Data Engineer
Sr Data Engineer Owings Mills, MD Any Visa Buying -75 Hybrid Job Description Compass Pointe has partnered with a global financial company in the Owings Mills, MD area that is looking to hire a Sr Data Engineer.
This position will focus on developing various database and data loading/transformation efforts.
Initiatives could include multiple source files and/or files sourced both internally and externally.
Candidate is required to interact with databases using AWS Lambda, AWS Glue (for transformations) Python and Spark.
Possible work leveraging SnapLogic for ETL, depending upon use case.
Work will be applicable to multiple applications.
Knowledge of automated testing of data a plus.
This role will be worked on a Hybrid schedule in Owings Mills, MD (2 days onsite, 3 days remote.
Must be onsite day 1)
Responsibilities:
Design and build systems for analytical reporting.
Analyze data and design ETL frameworks using Python and Spark technologies.
Collaborate with business partners to gather requirements and translate that to technical specifications.
Adhere to lean agile methodologies.
Production Support including trouble shooting and problem-solving capabilities.
Implement data pipelines to meet business requirements and adhere to best practices.
REQUIREMENTS:
10
years in a similar role.
Proficient in Python, Spark, and database technologies.
Experience with various AWS services such as AWS Glue, IAM, RDS, S3, SQS, SNS, Lambdas, Cloud Formation, Cloud Watch, Event Bridge, CLI.
Experience with Unix/Linux and SQL.
Experience with ETL.
Experience with source code control, preferably Git.
Experience in CI (Continuous Integration)/ CD (Continuous Delivery) software development pipeline stages like Commit, Build, Automated Tests, and Deploy.
DESIRED SKILLS:
Experience in creating AWS Glue components - Jobs, Triggers, Workflows.
Experience with AWS CLI.
Experience working with databases such as Snowflake and/or Aurora.
Experience with Hashicorp tool suite - Terraform, Vault.
Experience with soap and rest services.
Experience working with XML, JSON, YAML.
Experience with logging tools such as Cloudwatch and Splunk.
Experience with Snap logic.
Recommended Skills Agile Methodology Amazon Relational Database Service Amazon Simple Notification Service (Sns) Apache Spark Business Requirements Cloudformation Estimated Salary: $20 to $28 per hour based on qualifications.
This position will focus on developing various database and data loading/transformation efforts.
Initiatives could include multiple source files and/or files sourced both internally and externally.
Candidate is required to interact with databases using AWS Lambda, AWS Glue (for transformations) Python and Spark.
Possible work leveraging SnapLogic for ETL, depending upon use case.
Work will be applicable to multiple applications.
Knowledge of automated testing of data a plus.
This role will be worked on a Hybrid schedule in Owings Mills, MD (2 days onsite, 3 days remote.
Must be onsite day 1)
Responsibilities:
Design and build systems for analytical reporting.
Analyze data and design ETL frameworks using Python and Spark technologies.
Collaborate with business partners to gather requirements and translate that to technical specifications.
Adhere to lean agile methodologies.
Production Support including trouble shooting and problem-solving capabilities.
Implement data pipelines to meet business requirements and adhere to best practices.
REQUIREMENTS:
10
years in a similar role.
Proficient in Python, Spark, and database technologies.
Experience with various AWS services such as AWS Glue, IAM, RDS, S3, SQS, SNS, Lambdas, Cloud Formation, Cloud Watch, Event Bridge, CLI.
Experience with Unix/Linux and SQL.
Experience with ETL.
Experience with source code control, preferably Git.
Experience in CI (Continuous Integration)/ CD (Continuous Delivery) software development pipeline stages like Commit, Build, Automated Tests, and Deploy.
DESIRED SKILLS:
Experience in creating AWS Glue components - Jobs, Triggers, Workflows.
Experience with AWS CLI.
Experience working with databases such as Snowflake and/or Aurora.
Experience with Hashicorp tool suite - Terraform, Vault.
Experience with soap and rest services.
Experience working with XML, JSON, YAML.
Experience with logging tools such as Cloudwatch and Splunk.
Experience with Snap logic.
Recommended Skills Agile Methodology Amazon Relational Database Service Amazon Simple Notification Service (Sns) Apache Spark Business Requirements Cloudformation Estimated Salary: $20 to $28 per hour based on qualifications.
|
|
 | |
