Example Company
Software engineer
- Location
- San Francisco
- Type
- Full time
- Compensation
- $150k-$200k
- Posted
- 2 days ago
Join a team building useful software. Work closely with engineering and design to turn new ideas into products.
Loading job details
Software Engineer III- Data Engineer(Python, Databricks and AWS)
- Type
- Full time
- Posted
- 19D AGO
About the role
Job Description:
As a Data Engineer III at JPMorganChase within the Corporate Sector you serve as a seasoned member of an agile team to design and deliver trusted data collection, storage, access, and analytics solutions in a secure, stable, and scalable way. As a Data Engineer, you will be responsible for designing, building, and maintaining robust data pipelines and architectures that support the entire data engineering lifecycle. You will work closely with cross-functional teams to deliver scalable solutions that drive business insights and enable advanced analytics, including AI initiatives.
Key Responsibilities:
- Design, develop, and optimize end-to-end data pipelines using Databricks, Apache Spark, and AWS cloud services.
- Implement and manage workflow orchestration with Airflow.
- Integrate and process streaming data using Kafka.
Uses enterprise-authorized AI capabilities within the work environment to accelerate data pipeline/design analysis and documentation, validating outputs and handling data according to sensitivity and security requirements.
- Collaborate with data scientists, analysts, and business stakeholders to understand requirements and deliver high-quality data solutions.
- Ensure data quality, integrity, and security across all stages of the data lifecycle.
- Develop and maintain ETL processes, data models, and data warehouses.
- Write efficient, maintainable code in Python and SQL for data processing and analysis.
- Applies reuse-first, AI-assisted practices to strengthen SDLC-quality routines for data pipelines (e.g., test generation and control validation), ensuring traceability/auditability and alignment to resiliency and security expectations.
- Support AI and machine learning initiatives by preparing and engineering data for model development and deployment.
- Monitor, troubleshoot, and optimize data workflows for performance and reliability.
- Stay current with industry best practices and emerging technologies in data engineering and AI.
Required qualifications, capabilities, and skills
- Formal training or certification on data engineering concepts and 3+ years applied experience
- Proven experience in the full lifecycle of data engineering, from ingestion to production.
Demonstrated experience using enterprise-authorized AI capabilities within the work environment to support data engineering workflows with strong validation habits and awareness of data sensitivity.
Ability to review and validate AI-assisted outputs (e.g., query suggestions, test ideas, or model change summaries) before use, escalating when uncertain and following data handling requirements.
- Strong hands-on expertise with Databricks, Apache Spark, AWS (S3, Redshift, Lambda, etc.), Airflow, and Kafka.
- Proficient in Python programming and SQL for data manipulation and analysis.
- Experience with data modeling, ETL development, and data warehousing.
- Familiarity with AI/ML workflows and supporting data pipelines for machine learning.
- Certifications in relevant technologies (AWS, Databricks, Spark, etc.) are highly desirable.
- Excellent problem-solving skills and attention to detail.
- Strong communication and teamwork abilities.
Preferred qualifications, capabilities, and skills
- Experience with containerization (Docker, Kubernetes).
- Knowledge of data governance and compliance standards.
- Exposure to big data tools and frameworks.
- Ability to work in a fast-paced, agile environment.
