Senior DataBricks - Freelance

If you are a Databricks Developer then I have a fantastic opportunity for you.

The role is completely remote

The candidate would be ideally based in Europe

6 months +

Job Description:

We are seeking a highly skilled Senior Databricks Developer to join our team. The ideal candidate will have extensive experience with Databricks in an AWS environment, strong expertise in data integration and pipeline development, and proficient Python development skills. This role involves designing, developing, testing, deploying, maintaining, and improving data integration pipelines, along with advanced SQL and analytical tasks.

Key Responsibilities:

Databricks Development:

* Utilize Databricks to build and optimize large-scale data processing systems in an AWS environment.

* Implement and maintain data integration pipelines using Databricks, ensuring robust and efficient data processing.

AWS Cloud Integration:

* Integrate Databricks with various AWS services such as Apache Spark, Glue, Kinesis, and S3.

* Design and implement data solutions that leverage the scalability and flexibility of AWS cloud infrastructure.

Python Development:

* Develop robust data processing scripts and applications using Python and common Python libraries.

* Maintain and enhance existing Python codebases, ensuring high performance and reliability.Data Integration and Pipeline Development:
* Design, develop, test, deploy, maintain, and improve data integration pipelines.
* Ensure pipelines are efficient, scalable, and maintainable, and align with best practices.

SQL and Analytical Tasks:

* Write complex SQL queries, optimize query performance, debug issues, and manage database objects such as user-defined functions, views, and indexes.

* Perform detailed analysis on data, providing insights and recommendations based on findings.

Source Control and Continuous Integration:

* Use source control systems such as GitLab for version control of codebases.

* Implement continuous integration and deployment processes using Jenkins or similar tools to ensure smooth and automated code deployment.

Experience:

* Strong experience with Databricks in an AWS environment.

* Proven experience in data integration and pipeline development.
* Extensive experience with AWS services such as Apache Spark, Glue, Kinesis, and S3.
* Strong real-life experience in Python development, including familiarity with common Python libraries.
* Advanced SQL skills, including query optimization, debugging, and managing database objects.

Technical Skills:

* Proficiency in Databricks and AWS cloud services.

* Expertise in designing and implementing data integration pipelines.
* Strong analytical skills and experience writing complex SQL queries.
* Familiarity with source control systems such as GitLab and continuous integration tools like Jenkins.

Please do not hesitate to contact me on the details below if this is of interest to you.