Big Data Developer

Description:

Mindlance is seeking a talented and experienced Big Data Developer to join our growing team. The ideal candidate will possess a strong background in building robust and scalable data processing pipelines and systems. You will be instrumental in designing, developing, and implementing solutions for ingesting, storing, processing, and analyzing large datasets. Your expertise will contribute to extracting valuable insights and driving data-informed decisions within the organization.

Key Responsibilities:

• Design, develop, and maintain scalable big data processing pipelines and frameworks.
• Build and optimize data storage solutions utilizing technologies such as Hadoop, Spark, NoSQL databases, and data warehouses.
• Implement data ingestion, transformation, and processing workflows for various data sources.
• Develop and deploy data analytics and machine learning models at scale.
• Ensure data quality, integrity, and security throughout the data lifecycle.
• Write clean, efficient, and well-documented code following best practices for big data development.
• Troubleshoot and resolve issues related to data processing and performance in production and development environments.
• Collaborate with data scientists, analysts, and other engineering teams to understand data requirements and deliver effective solutions.

Required Skills:

• Strong programming experience in languages such as Java, Scala, or Python, with a focus on big data processing.
• Proven experience with big data technologies and frameworks (e.g., Hadoop, Spark, Kafka, Flink).
• Hands-on experience with NoSQL databases (e.g., Cassandra, MongoDB, HBase) and data warehousing solutions (e.g., Snowflake, Redshift).
• Experience with data ingestion and ETL/ELT tools and processes.
• Familiarity with cloud platforms (e.g., AWS, Azure, GCP) and their big data services.
• Understanding of data modeling, data warehousing concepts, and data governance principles.
• Excellent problem-solving and analytical skills with the ability to work with large and complex datasets.