职位&公司对比

职位详情

• 上海
• 5-10年
• 本科

• Java
• 大数据经验
• 不接受居家办公
• Hive
• 分布式经验
• Hadoop
• Spark

Company Description eBay is a technology company driving the ‘future of commerce’! We are enabling consumers to buy anything, anywhere, anyhow, thus unifying the online buying experience across the mobile, local, social and digital space. Job Description We leverage our data & AI technology to accelerate growth, introduce new revenue streams, and create transformative experiences for our customers. Our platform deals with petabytes of data that is processed, queried, visualized, analyzed, with high quality & low latency. We are seeking for big data expert to buil直聘d on LLM-empowered data platform to enable data & AI. Responsibilities ● Build Data Mesh that faci直聘litate seamless data integration across different environmebossnts, such as APIs, data virtualization, and integration platforms like Apache NiFi or Talend. ● Design Event-Driven Architectures: Experience in designing and implementing event-driven architectures that support real-time data flows. ● Develop Data Orchestration: Knowledge of orchestration tools such as Apache Airflow or Prefect to automate and manage data workflows. ● Design and implement self-service data infrastructure that empowers teams to manage their data independently. ● Familiarity with open直聘 standards and protocols to ensure data interoperability across domains. ● Desig直聘n Metadata Management which proficiency in managing and utilizing metadata to enhance data discovery, lineage, and governance. Familiarity with tools like Apache Atlas or Collibra is beneficial. Qualifications ● Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field, with 5+ years of experience in big data engineering or a similar role. ● Strong proficiency in Java and Spring Boot for developing and deploying data applications. ● Extensive experience with Apache Spark and Apache Flink for both batch and stream data processing. ● Experience with open source solutions like DBT, Sql Mesh is a plus. ● Familiarity with metadata management, data linkage, and data governance frameworks to ensure data integrity and knowledge discovery is a plus ● Experience with Redis and MongoDB, including expertise in database indexing for efficient data consumption is a plus. ● Proficiency in Python, particularly for integrating and deploying machine learning models, including LLMs prompt engineering is a plus. ● Communication in English is required.