Pyspark Engineer IRC253379

Company: Hitachi Vantara Corporation
Location: San Jose
Posted on: February 1, 2025

Job Description:

Description:Retail giant in North America.Requirements:Technical Skills:

• Expertise in PySpark, Spark SQL, DataFrames, and UDF.
• Strong proficiency in Python with experience in data structures, algorithms, and OOP.
• Experience with large-scale distributed data processing and parallel computing.
• Working knowledge of Apache Kafka for real-time streaming applications.
• Strong familiarity with cloud-based S3 or MinIo bucket.
• Knowledge of SQL for querying large data sets and working with relational databases.
• Knowledge of database queries optimization.
• Strong experience on processing large PySpark dataset on executors node and parallelism operations between executors and driver node.
• Collaborate with cross-functional teams to develop applications that meet business requirements and quality software.Tools & Frameworks:
  • Proficiency in data processing frameworks like Hadoop.
  • Familiarity with data orchestration tools such as Apache Airflow and Spark Compute platform.
  • Good experience with development IDE PyCharm.Job Responsibilities:
    • Collaborate with cross-functional teams to develop applications that meet business requirements and quality software.What We Offer:Exciting Projects: Come take your place at the forefront of digital transformation! With clients across all industries and sectors, we offer an opportunity to work on market-defining products using the latest technologies.Collaborative Environment: You can expand your skills by collaborating with a diverse team of highly talented people in an open, laidback environment - or even abroad in one of our global centers or client facilities!Work-Life Balance: GlobalLogic prioritizes work-life balance, which is why we offer flexible work schedules and opportunities to work from home.Professional Development: We provide continuing education classes, professional certification and training (technical, soft skills, language, and communication skills) to help you realize your professional goals. Being part of a global organization, there are additional learning opportunities through international knowledge exchanges.Excellent Benefits: We provide our employees with competitive salaries, health and life insurance, short-term and long-term disability insurance, a matched contribution 401K plan, flexible spending accounts, and PTO and holidays.About GlobalLogic:GlobalLogic is a leader in digital engineering. We help brands across the globe design and build innovative products, platforms, and digital experiences for the modern world.By integrating experience design, complex engineering, and data expertise, we help our clients imagine what's possible and accelerate their transition into tomorrow's digital businesses.Headquartered in Silicon Valley, GlobalLogic operates design studios and engineering centers around the world, extending our deep expertise to customers in the automotive, communications, financial services, healthcare and life sciences, manufacturing, media and entertainment, semiconductor, and technology industries.GlobalLogic is a Hitachi Group Company operating under Hitachi, Ltd. (TSE: 6501), which contributes to a sustainable society with a higher quality of life by driving innovation through data and technology as the Social Innovation Business.
      #J-18808-Ljbffr

Keywords: Hitachi Vantara Corporation, Cupertino , Pyspark Engineer IRC253379, Engineering , San Jose, California