Product Development

Data Engineer - Spark/Java/Python

Pune, Maharashtra
Work Type: Full Time

Location: Hyderabad/Bengaluru

You will work on:
As part of our client’s solution development team, you will work closely with product, sales, marketing, and customer services teams to design, build and operate an enterprise cloud-based product solution. As Senior Lead or Architect, you will have the opportunity to lead a collaborative and agile team to deliver industry-leading solutions to Fortune 100 companies.

What you will do (Responsibilities):

  • Collaborate with product management & engineering to build/maintain highly efficient data pipelines for large datasets.
  • Perform Data quality analysis on disparate data sources and define, implement data quality rules
  • Design and develop Spark data pipelines analyzing end to end data requirements
  • Perform impact analysis & upgrades of existing Hive/Spark Data Pipelines
  • Troubleshooting data loss, data inconsistency, and other data-related issues
  • Product development environment delivering stories in a scaled agile delivery methodology.

What you bring (Skills):

  • 2-4 year of experience in hands-on data engineering & medium scale distributed applications
  • Good experience in object-oriented programming languages such as Java or Python
  • Sound experience in RDBMS such as MySQL, Oracle, SQLServer, etc.
  • Strong Experience in big data processing technologies such as Hadoop, Spark, Kafka  etc.
  • Strong Experience in developing, maintaining and deploying batch data pipelines in Hive, Spark
  • Experience with Scrum and/or other Agile development processes
  • Strong analytical and problem-solving skills
  • Team player with self-drive to work independently
  • Strong communication and interpersonal skills

Great if you know (Skills):

  • Experience in developing streaming data pipelines using Apache Flink
  • Experience in Cloud appliances as Snowflake, Redshift, BigQuery, Azure Synapse etc
  • Experience in Cloud-based services such as Amazon AWS, Microsoft Azure, or Google Cloud Platform
  • Some exposure to containerization technologies such as Docker, Kubernetes, or Amazon ECS/EKS
  • Some exposure to NoSQL data stores such as Couchbase, Solr, etc.
  • Ability to learn new technologies on his/her own, perform POCs
About Cognologix:

Cognologix helps companies disrupt by reimagining their business models and innovate like a Startup. We are at the forefront of digital disruption and take a business first approach to help meet our client’s strategic goals.

We are Product development focused organization helping our clients build their next-generation products & services.

Minimum Experience:
2+ Years
Top Skill:
Data Engineer, Spark, Java, Python, MySQL, Haddop, Kafka,Hive

Submit Your Application

You have successfully applied
  • You have errors in applying