Lead, Big Data Engineer

Das ist der Job

Responsibilities: Build the Unified Asset Management product in the cloud.

Darum lohnt es sich

Job Description: Lead Big Data Engineer responsible for delivering roadmap features of Unified Asset Inventory, building highly scalable micro‑services interacting with the Qualys Cloud Platform, researching, evaluating and adopting next‑generation technologies, and producing high‑quality software following good architecture and design principles.

Design and develop highly scalable micro‑services that interact with the Qualys Cloud Platform. Research, evaluate, and adopt next‑generation technologies. Produce high‑quality software following good architecture and design principles. Work on challenging and business‑impacting projects.

Qualifications: Bachelor’s degree in computer science or equivalent. 10+ years of total experience. 4+ years of relevant experience in designing and architecting Big Data solutions using Spark. 3+ years of experience working with engineering resources for innovation. 4+ years of experience understanding Big Data event flow pipelines. 3+ years of experience in performance testing for large infrastructure. 3+ years of in‑depth experience understanding various search solutions (Solr/Elastic). 3+ years of experience in Kafka.

In‑depth experience with data lakes and related ecosystems. In‑depth experience with message queueing. In‑depth experience giving requirements to build scalable architecture for Big Data and micro‑services environments. In‑depth experience understanding caching components or services. Knowledge of Presto technology. Knowledge of Airflow.

Hands‑on experience in scripting and automation. In‑depth understanding of RDBMS/NoSQL (Oracle, Cassandra, Kafka, Redis, Hadoop) and lambda, kappa, kappa++ architectures with Flink data streaming and rule engines. Experience with machine learning model engineering and related deployment.

Design and implement secure big data clusters to meet many compliances and regulatory requirements. Experience leading delivery of large‑scale systems focused on managing the infrastructure layer of the technology stack. Strong experience in performance benchmarking testing for Big Data technologies. Strong troubleshooting skills.

Experience leading development life cycle processes and best practices. Experience in Big Data services administration is a plus. Experience with Agile Management (SCRUM, RUP, XP), object‑oriented modeling, and working on Internet, UNIX, middleware, and database related projects.

Experience mentoring/training the engineering community on complex technical issues. #J-18808-Ljbffr