Overview
As a Python Developer, you will provide solutions on the data processing, engineering, and enrichment phase for various projects. You will be responsible for some aspects of the data life cycle including design, maintenance and building of cloud and data architectures. These activities will deliver on ingestion, transformation, and building of high-quality data structures.
What will you do at dyvenia?
You will be responsible for:
- Creating, monitoring, and troubleshooting ETL jobs to ingest and transform data from various systems using batch and stream processing;
- Driving and advocating for state-of-the-art methods of handling data in the company, eg. data lake, stream processing, IaC, DevOps, DataOps
- Collaborate with analytics and business teams to improve data feed.
- Designing, developing, and maintaining the current code base
- Assisting in team leadership, coordinating requests, addressing and creating user stories
As a DE, the majority of your work will involve performing new data ingestions from various sources such as databases, APIs, and files. This will require your skills in extracting, loading, and transforming data into a usable format for downstream analysis.
The remaining of your work will involve supporting the maintenance of existing ETL jobs. This will require your attention to detail and strong problem-solving skills in monitoring, troubleshooting, and resolving any issues that arise.
Your role will be critical in ensuring the successful ingestion and processing of new data, while maintaining the integrity and efficiency of existing ETL jobs.
You will be working with the following technologies:
- Python, Prefect (preferred), SQL
- Message and/or task queues
- Microsoft Azure
- Docker
- Kubernetes
- Linux, bash, git
- CI/CD
You will be based at our office in Krakow in a hybrid format.
What will you need to be successful in the role?
- Demonstrated proficiency in the data engineering area
- Intermediate Python
- Strong SQL
- Understanding of Docker
- Familiarity with Linux
- Experience with ETL
- Experience in creating APIs using Python backend
- Familiarity with Postgres database
- Knowledge of message brokers (Kafka, Redis Streams, RabbitMQ, MQTT) or task queues (RQ, arq, Celery)
Benefits
When you join our team, you can expect:
Trainings
We want you to thrive as a member of our team and for that reason, we organize internal trainings and provide you with a training budget.
Hands-on mentorship
At the beginning of your journey, you are guided by a mentor who supports you through your career development and helps you find ways to grow professionally.
Flexibility
We provide flexible working hours and possibility to work from home (days per week vary during probation period and based on seniority).
Team events
We truly enjoy spending time together and getting to know one another better during various team-building activities.
Medical healthcare
We geniuenely care about your wellbeing and therefore we provide private healthcare for you and your family.
Sport packages
We offer sport packages to support your healthy lifestyle and help you improve in your favorite sport activities.
What are the next steps?
- Apply to this job by sending us your CV at [email protected]
- We get back to everyone, but if you don’t hear from us within a week, send us a quick reminder
- The very best of luck!