This job is located in Jacksonville, FL and you'd need to report to that office daily, but would consider applicants that are in the greater Atlanta, GA area as well.
Job Summary: The Soliant Data Engineer designs, builds, and maintains scalable, secure, and highly reliable data pipelines within Soliant’s Azure-based data platform. This role is heavily focused on Databricks development, leveraging Delta Lake and Spark to transform raw data into trusted, analytics-ready datasets that support enterprise reporting, financial analytics, and operational insights.
The Data Engineer partners closely with analytics, finance, and business teams to ensure data is accurate, timely, and well-governed across the organization.
Key Responsibilities:
Databricks & Data Pipeline Development
- Design, develop, and maintain scalable data pipelines using Azure Databricks, Delta Lake, and Apache Spark
- Implement batch and incremental data processing patterns to support enterprise analytics use cases
- Write efficient, well-structured PySpark and SQL code following engineering best practices
Data Ingestion & Orchestration
- Build and orchestrate data ingestion workflows using Azure Data Factory, integrating with Databricks notebooks and jobs
- Ingest data from APIs, flat files, and relational data sources into Azure Data Lake Storage (ADLS Gen2)
Data Transformation & Modeling
- Transform raw data into curated, analytics-ready datasets using Delta Lake (Bronze / Silver / Gold) patterns
- Apply data quality checks, deduplication logic, and validation rules to ensure high data reliability
- Support downstream reporting and semantic models used in Power BI
Data Architecture & Storage
- Collaborate with the team to design scalable data architectures that support current and future business needs
- Manage structured and semi-structured data stored in Azure Data Lake and Azure SQL Database
- Ensure data storage solutions are secure, performant, and cost-effective
Automation, Optimization & Reliability
- Optimize Databricks workloads for performance and cost (partitioning, caching, job tuning)
- Automate data processing and operational tasks using Azure services and scripting tools
- Monitor pipelines and jobs, troubleshoot failures, and continuously improve reliability and performance
Collaboration & Stakeholder Engagement
- Work closely with financial analysts, data enrichment analysts, and reporting teams to understand data requirements
- Provide clean, trusted datasets and support ad-hoc analytical needs when required
Required Skills & Qualifications:
- Bachelor’s degree in Computer Science, Data Engineering, Information Systems, or a related field
- Strong experience building data pipelines in Microsoft Azure, with a focus on Azure Databricks
- Hands-on experience with:
- Azure Databricks
- Azure Data Factory
- Azure Data Lake Storage (ADLS Gen2)
- Azure SQL Database / SQL Server
- Strong proficiency in SQL and Python (PySpark)
- Experience with ETL/ELT patterns and data transformation in a cloud environment
- Solid understanding of data warehousing concepts and analytical data modeling
- Experience monitoring, troubleshooting, and optimizing production data pipelines
- Strong problem-solving skills, attention to detail, and collaboration abilities
Nice to Have:
- Experience with Delta Lake and medallion architectures (Bronze / Silver / Gold)
- Familiarity with CI/CD pipelines for data engineering (Azure DevOps, GitHub Actions, or similar)
- Experience using Git for source control and collaborative development
- Exposure to Databricks Jobs, workflows, and cluster configuration
- Experience optimizing Spark workloads (partitioning strategies, file sizing, performance tuning)
- Familiarity with data governance, access controls, and security best practices in Azure
- Experience supporting Power BI semantic models or downstream reporting platforms
- Knowledge of infrastructure-as-code concepts (ARM, Bicep, Terraform)
Why Soliant
- Modern Azure data platform with Databricks at the core
- Opportunity to build and improve enterprise-scale data pipelines
- High-impact work supporting financial, operational, and executive reporting
- Collaborative environment with strong ownership and autonomy
