Azure Data Engineer

Summary
As an Azure Data Engineer at Gainwell, you will be a part of an innovative healthcare technology company with a mission to serve the healthcare needs of our communities.
Primary skills: ( Must have 5 or more years experiene in ADF, SSMS, SQL, PowerBI )
Candidates should possess skills in
stylemargin:bottom:11.0px:
:
ADF pipeline set up and management
:
SQL programming (MS SQL Server : SSMS)
:
Issue Resolution and Troubleshooting in Azure environments
:
Testing and validations
:
Automation activities
:
Set up ETL activities in ADF framework
:
Identify process efficiencies opportunities
Secondary skills:
stylemargin:bottom:11.0px:
:
Analyze logs and set up monitoring mechanisms
:
Security and Access Management
:
Problem solving and root cause analysis
Candidate should have expertise in:
stylemargin:bottom:11.0px:
:
Azure Data Factory, Azure Data Lake, and Azure SQL Database
:
Azure DevOps (for managing CI/CD processes, including pipeline setup, tracking, and performance monitoring)
:
Working knowledge of data integration patterns and ETL processes
Key Responsibilities:
Data Pipeline Management:
stylemargin:bottom:11.0px:
:
Build and execute ADF pipelines for continuous integration/continuous deployment (CI/CD) processes.
:
Set up and configure multi:stage pipelines and various environment configurations for development, testing, and production.
:
Track pipeline performance and ensure the health of applications, taking corrective actions when needed.
:
Design processes for monitoring pipeline activities and system health.
:
Integrate data from multiple sources, including SQL databases and third:party APIs.
:
Develop data integrations between source and destination systems to support operational and reporting needs.
:
Build infrastructure for data extraction, transformation, and loading (ETL) using MS SQL and Azure technologies.
:
Identify opportunities for automating tasks within the ADF framework and execute enhancements to improve efficiency.
:
Set up Azure Monitor to track performance metrics, logs, and alerts for Azure resources.
SQL Operations:
stylemargin:bottom:11.0px:
:
Run SQL programs within Azure environments, optimizing for performance and data accuracy.
:
Work with data in relational databases and assist with query optimization and troubleshooting.
Issue Resolution and Troubleshooting:
stylemargin:bottom:11.0px:
:
Perform root cause analysis and debugging to address issues related to application performance.
:
Coordinate with the Azure IT team to troubleshoot memory leaks, inefficient memory usage, and other performance bottlenecks in applications.
:
Provide rapid response to resolve critical system errors and ensure minimal downtime.
Logs and Monitoring:
stylemargin:bottom:11.0px:
:
Create and manage rules and alerts for system monitoring.
:
Analyze logs from various Azure resources to detect and resolve performance issues.
:
Set up comprehensive logging and monitoring solutions for CI/CD processes to ensure system reliability.
Testing and Validation:
stylemargin:bottom:11.0px:
:
Conduct thorough testing and validation of pipelines and infrastructure before production deployment.
:
Collaborate with the IT and development teams to perform functional, performance, and regression testing.
Security and Access Management:
stylemargin:bottom:11.0px:
:
Coordinate with IT teams to manage access control, granting or revoking access as needed and raising tickets for user management.
:
Maintain documentation of current pipelines, refresh processes, and security configurations.
:
Coordinate and set up various security levels in reports and applications to ensure data privacy and compliance with security protocols.
:
Assist in the review and audit of Azure infrastructure architecture, providing insights for improvements and op
Job Type: Permanent