Hpc Data Engineer
CurrentCollect and provide access to telemetry and operations data from the ORNL supercomputers and their supporting systems. • Develop high-scale streaming message platforms with an operational focus• Implementing robust end-to-end data processing pipelines• Providing systems administration staff and other data owners with the tools and knowledge to successfully extract necessary data from systems• Interfacing with research teams to understand data needs, and providing subject matter expertise to research teams and other team members about the data models, query optimizations, and schema interpretation• Designing and developing architectures for intake, curation, organization, and dissemination of data in support of data science and related disciplines• Researching and evaluating the state-of-the-art data and information management technologies related to Big Data• Working on a variety of data assignments; collaborate with scientists and engineers, and expect to produce reliable data products and systems• Documenting data dictionaries, schemas, and paths and maintaining a catalog of available data sources