Senior Python Data & AI Engineer (Web Crawling) - Project setup requires service delivery from a nea

ResponsibilitiesAnalysis and Indexing of Appropriate Web Sources: The contractor shall identify and evaluate appropriate sourcesDesign of the crawling and extraction solution: Development of a technical target architectureDevelopment of Crawling, Extraction, and AI ComponentsData Structuring and Provision: Transformation of the extracted informationQuality Assurance and Domain Validation: Implementation of procedures and a Databricks app Pilot TestingDocumentation and HandoverProfileWeb Crawling & Web Scraping: Scrapy, Beautiful Soup, or comparable frameworks, HTML, DOM, and table analysis, working with dynamic web pages and download structuresInformation Extraction & Generative AI: LLM & NLP, prompt engineering and structured output formats, classification, mapping, and duplicate detectionInterface and Document Processing: Rest APIs, PDF and datasheet extraction, Table extraction and

schema mappingData Engineering & Databricks, Python, Pandas and PySpark, Delta Lake, and development of Databricks appsBenefitsA very renowned companyPossibilities to work remote