Title: Data Engineer #00246
State Role Title: Information Tech Spec III
Hiring Range: $100,000 - $125,000
Pay Band: 6
Agency: The Library of Virginia
Location: Library of Virginia Building
Agency Website: https://www.lva.virginia.gov
Recruitment Type: General Public - G
Job Duties
The Library of Virginia seeks a versatile and motivated Data Engineer to help reimagine, transform, and administer an improved data platform and information infrastructure, from ingest through access. This redesigned role will have the opportunity to collaboratively transform data practices at a 200-year-old cultural heritage organization and state agency with an eye towards the future.
The work will create efficiencies; improve staff information gathering, efficiency, and productivity; and enhance the Library’s data storage capacity. It will also support the development of applications that allow LVA staff and users to contribute to and interact with data and databases. The incumbent will manage security, optimize performance, and guide the evolution of legacy systems to meet changing and more complex modern requirements. They will perform database administration on new and existing systems across various platforms from design through performance tuning, data security, backup/recovery, and user management.
The ideal candidate will have broad, intermediate-to-advanced data engineering / database administration knowledge, with a strong background in data architecture and planning. They will have experience and comfort both leading and implementing technological change, stewarding projects from inception to completion. The incumbent will have a proven record in data pipeline processes and integration tools to collect, process, store, and deliver data from multiple formats in organized, accessible, user-friendly ways. They will have strong troubleshooting skills, the ability and desire to learn new technologies quickly, and experience collaborating with non-technical staff to translate their needs into actionable technical tasks. They will be comfortable supporting disparate applications and systems—multitasking on various initiatives— and will show a demonstrated ability to work collaboratively within a team.
Position Requirements:
· Technical Responsibilities:
· Perform initial and ongoing evaluations of existing data and database infrastructure with an eye towards modernization, sustainability, and customer service. Implement recommended changes as necessary.
· Ingest, process, transform, store and securely and accessibly deliver data for information sharing and decision-making.
· Analyze, design, develop, implement and maintain custom databases and other systems that collect, store, and process data efficiently.
· Migrate and combine large datasets from multiple formats/locations.
· Monitor databases and related systems to ensure optimized performance, data quality and security.
· Implement processes to validate data, ensuring accuracy and reliability.
· Document processes related to database design, configuration, and performance.
· Work closely with project managers, stakeholders and other technical staff to understand data requirements and ensure that the data infrastructure meets those needs.
· Review and analyze staff work procedures to identify strategies for automation.
· Envision, design and implement new systems to meet immediate and long-term needs of Library stakeholders.
· Serve as a database administrator, at present primarily MS SQL but with the possibility to change based on assessments.
· Assist with data conversion efforts, which includes working with developers to convert legacy access applications into a modern web format and working with external government entities to ingest custodial record transfers.
· Manage database upgrades for all SQL based systems (MS SQL 2016/2019/2022 [most prevalent], MySQL, PostgreSQL 15), as well as supporting system administration.
· Maintain and ultimately convert legacy Microsoft Access applications.
· Determine security measures to be taken to protect data integrity. This includes vulnerability remediation and working with internal/external auditors to ensure compliance with Virginia security standards.
· Adhere to best practices in securely backing up and archiving data.
· Work with ISO and system admins to ensure adequate data backups are made.
· Keep abreast of developments and best practices in data engineering.
• Skills and Applications:
· Strong experience with data architecture and planning.
· Experience with data analysis with understanding of data pipeline processes.
· Ability to quickly evaluate and understand legacy systems and unknown technologies.
· Experience using structured, semi-structured and unstructured data together for analysis and presentation.
· Experience tracking and managing changes through version control for enhanced collaboration, recovery, project management and traceability. Git knowledge a plus.
· Experience with MS SQL, MySQL.
· Working knowledge of PostgreSQL and Oracle a plus.
· Understanding of MS Access as a front-end interface.
· Proficiency in languages such as Python and familiarity with tools and frameworks like Apache Spark.
· Experience with integration tools.
· Familiarity with Microsoft data management suite (Purview, Azure, Dynamics, etc).
· Experience with performance tuning.
· Ability to recover lost or inaccessible data due to accidental deletion, file corruption, HW failure or malicious intent, restoring data integrity and ensuring usability.
· Working knowledge of multimodal RDBMS.
· Understanding of cloud platforms and storage methodologies.
· Strong technical leadership and guidance.
· Ability to formulate and explain a long-term tech stack goal.
· Considerable general database and application software knowledge.
· Software installation and troubleshooting ability.
· Strong ability to work independently and resolve problems of a complex nature.
· Customer service experience collaborating with internal and external stakeholders, including prioritization, balancing user needs with technical realities, managing conflicting user goals and translating technical concepts for non-technical audiences.
· Ability to manage time and multiple priorities effectively.
· Ability to communicate clearly both orally and in writing with non-technical users.
• Education and Expertise
· Considerable experience in data engineering or a comparable role.
· Any applicable licensing or certifications.
· In-depth knowledge of SQL.
· Extensive experience with database technologies and architecture.
· Practical knowledge of best practices in data engineering and data security.
· Strong organizational skills and attention to detail.
· Exceptional problem-solving and critical thinking skills.
· Excellent collaboration and communication skills.
Minimum Qualifications
Demonstrated expertise in
· data engineering
· data architecture
· data planning
· data pipeline processes
· data security
· SQL (particularly MS SQL, MySQL, or both)
· python
• Demonstrated experience using
· structured, semi-structured and unstructured data together for analysis and presentation
· version control for enhanced collaboration, recovery, project management and traceability
· data processing tools such as (but not necessarily) Apache Spark
· data integration tools such (but not necessarily) Azure Data Factory
· cloud platforms and storage methodologies
• Demonstrated ability to
· quickly evaluate and understand legacy systems and unknown technologies
· recover lost or inaccessible data due to accidental deletion, file corruption, HW failure, or malicious intent
· formulate, explain, and follow through on implementing long-term tech stack goals
· work and resolve complex problems independently
· Some familiarity with
· MS Access as a front-end interface
· Microsoft data management suite (Purview, Azure, Dynamics, etc.)
· Multimodal RDBMS
· Excellent writing, communication, and interpersonal skills required
Additional Considerations
· Experience working in cultural heritage, higher education, research (especially humanities), or similar sectors is a plus.
· Familiarity or expertise with the following are a plus:
o PostgreSQL
o Oracle
o Data performance tuning
o Microsoft data management tools
Special Instructions
You will be provided a confirmation of receipt when your application and/or résumé is submitted successfully. Please refer to “Your Application” in your account to check the status of your application for this position.
State employees who have been affected by Policy 1.3 Layoff and possess a valid Interagency Placement Screening Form (Yellow Card) or a Preferential Hiring Form (Blue Card) must submit the card BEFORE the closing date for this position. The card may be scanned and attached to the application or faxed to (804) 371-7401. Please include your name and the position number on the fax cover sheet. AMERICORPS, PEACE CORPS, AND OTHER NATIONAL SERVICE ALUMNI ARE ENCOURAGED TO APPLY.
Reasonable accommodations are available to individuals with disabilities during the application and/or interview processes per the Americans with Disabilities Act. Please call 804-225-2131 for assistance.
All applicants are subject to a background investigation. The investigation may include: fingerprint checks (State Police, FBI); local agency checks; employment verification; verification of education (relevant to employment); credit checks; and other checks requested by the hiring authority.
Equal Opportunity Employer
Contact Information
Name: Marjorie McGregor
Phone: 804-338-9034
Email: marjorie.mcgregor@dhrm.virginia.gov
In support of the Commonwealth’s commitment to inclusion, we are encouraging individuals with disabilities to apply through the Commonwealth Alternative Hiring Process. To be considered for this opportunity, applicants will need to provide their AHP Letter (formerly COD) provided by the Department for Aging & Rehabilitative Services (DARS), or the Department for the Blind & Vision Impaired (DBVI). Service-Connected Veterans are encouraged to answer Veteran status questions and submit their disability documentation, if applicable, to DARS/DBVI to get their AHP Letter. Requesting an AHP Letter can be found at AHP Letter or by calling DARS at 800-552-5019.
Note: Applicants who received a Certificate of Disability from DARS or DBVI dated between April 1, 2022- February 29, 2024, can still use that COD as applicable documentation for the Alternative Hiring Process.