Product Engineering

Senior Data Engineer(Databricks/Spark)

Pune, Maharashtra
Work Type: Full Time
You will work on:

We are looking for self driven Data Professional to be a key member of our Data Practice. We help many of our clients make sense of their large investments in data – be it building analytics solutions or machine learning applications. You will work on cutting edge cloud-native technologies to crunch terabytes of data into meaningful insights.

Job Location: Pune/Remote
 
What you will do (Responsibilities):

  • Collaborate with product management & engineering to build highly efficient data pipelines for large datasets.
  • Perform Data quality analysis on disparate data sources and define, implement data quality rules
  • Design and develop data models analyzing end to end data requirements
  • Troubleshooting data loss, data inconsistency, and other data-related issues
  • Product development environment delivering stories in a scaled agile delivery methodology
  • Create high quality project level technical artifacts such as requirement specs, high/detailed level design documents, test plans etc
  • Supporting in technical POCs, Presales activities for newer opportunities
  • Contributing to maintaining frameworks, guidelines, standards for applicable project technologies and strengthening engineering practices
  • Supporting research, POCs in emerging technologies
  • Mentoring associates, team members
What you bring (Skills):

  • 5+ year of experience in hands-on data engineering & medium to large-scale distributed applications
  • Sound understanding of data warehousing, Data Lake and Lakehouse concepts
  • Experience in big data processing technologies such as Databricks, Spark, Kafka etc.
  • Extensive experience in Python or object-oriented programming languages such as Java or Scala
  • Extensive experience in RDBMS such as MySQL, Oracle, SQLServer, etc and.or Cloud appliances as Snowflake, Redshift, BigQuery, Azure Synapse etc.
  • Experience in Cloud-based services such as AWS, Microsoft Azure, or Google Cloud Platform
  • Experience in developing and deploying applications in Linux OS
  • Experience with Scrum and/or other Agile development processes
  • Ability to create WBS for project tasks and contribute to planning
  • Strong analytical and problem-solving skills
  • Team player with self-drive to work independently
  • Strong communication and interpersonal skills

Great if you know (Skills):

  • Exposure to containerization technologies such as Docker, Kubernetes, or Amazon ECS/EKS
  • Exposure to NoSQL data stores such as Couchbase, Solr, etc.
  • Exposure shell scripting or orchestration tools like Airflow
  • Ability to lead R&D and POC efforts

Advantage Cognologix:

  • A higher degree of autonomy, startup culture & small teams
  • Opportunities to become an expert in emerging technologies
  •  Remote working options for the right maturity level
  •  Competitive salary & family benefits
  •  Performance based career advancement

About Cognologix:

Cognologix helps companies disrupt by reimagining their business models and innovate like a Startup. We are at the forefront of digital disruption and take a business-first approach to help meet our client’s strategic goals.

We are a Data focused organization helping our clients to deliver their next generation of products in the most efficient, modern, and cloud-native way.
Minimum Experience:
5+ Years
 
Top Skill:
Data Engineering, Data warehousing, Data Lake, Databricks,MySQL,Java, Scala,Cloud Platforms
 

Submit Your Application

You have successfully applied
  • You have errors in applying