Aniket Yadav

Lead Data Engineer @Taiyo 		Github @M0N0Atomic
Updated
Location Greater Noida, Uttar Pradesh Country India

About Me

I have been involved with computers since my childhood. Video games were my first introduction to computers and I have been hooked since then.

Later, I was introduced to Telegram bots and I was fascinated by the idea of automating things. I started learning Python and started building on top of other's work (never wrote one from scratch). This was the time when I was introduced to the world of Python and open-source. I continued to learn Python during my college days.

I started my career as a Data Engineer Associate at taiyo.ai. Initially working on the data pipelines to scrape data from the web and drafting documentation and data standards for the company. Scraping taught me a lot about the web and how it works: connections, headers, cookies, auth, CORS, bot-protection, REST-APIs etc. I was also introduced to Elasticsearch and Kibana. When I transitioned to Data Engineer, I was tasked with building backend service for SERP APIs and AI driven workflows. Currently, I am working on building AI Agents and Data Processing Pipelines.

I explore new technologies and tools in my free time and experiment with them.

Projects

Research: An AI-Driven Data Mesh Architecture Enhancing Decision-Making in Infrastructure Construction and Public Procurement

Source: arXiv:2412.00224

SERP API
Source NOT PUBLIC
Description API to fetch news, internet, etc. search results with lower latency as compared to commercial APIs.
Technologies Asynchronous Programming, Quart, Elasticsearch
Challenges Scraping, Bot Protection
Objective Deliver latest news and internet search results to the user and AI Agents in real-time.
AI Agent Framework
Source NOT PUBLIC
Description Python framework to build AI Agents for various tasks.
Technologies Python, Pydantic, OOP Design
Challenges ▪ Designing a generic framework to build AI Agents
▪ Making it LLM provider agnostic
▪ Consistent interface across LLMs
▪ Allow Agent instructions to be updated in realtime
Objective Build AI Agents for various tasks like risk analysis, report generation, information retrieval, etc. with ability modularity to switch between LLM and parameters, instructions in real-time.
AI Scraping Framework
Source NOT PUBLIC
Description Python framework to scrape data from the web using AI.
Technologies Python, Pydantic, Scrapy
Challenges ▪ Designing a generic framework to scrape data from the web
▪ Ensuring data standards and quality
Objective Scrape data from the web using AI and ensure data standards and quality, eliminating the need for manually writing scrapers for each website.

Experience

      Present 
         │
2023-10 ─┴─ Lead Data Engineer, Taiyo   
         │
2023-01 ─┴─ Data Engineer, Taiyo
         │
2022-10 ─┴─ Data Engineer Associate, Taiyo
         │
2021-08 ─┴─ Research Intern, Mumbai Rail Vikas Corporation
         │
2020-08 ─┴─ Research Intern, Indian Railways (Mumbai Division)

Skills & Tools

┌─────────────────────────┬────────────────────────────────────────────────┐
│ ┌─────────────────────┐ │    ┌────────────────┐ ┌───────────────────┐    │
│ │       Skills        │ │    │Data Engineering│ │Backend Engineering│    │
│ └─────────────────────┘ │    └────────────────┘ └───────────────────┘    │
|                         |    ┌───────────────┐ ┌────────┐                │
│                         │    │Task Automation│ │SysAdmin│                │
│                         │    └───────────────┘ └────────┘                │
│                         │    ┌──────────────────┐                        │
│                         │    │Prompt Engineering│                        │
│                         │    └──────────────────┘                        │
│ ┌─────────────────────┐ │    ┌──────┐ ┌──────┐ ┌───┐ ┌───┐               │
│ │Programming Languages│ │    │Python│ │Golang│ │Zig│ │ R │               │
│ └─────────────────────┘ │    └──────┘ └──────┘ └───┘ └───┘               │
│ ┌─────────────────────┐ │    ┌─────────────┐ ┌───────┐                   │
│ │      Databases      │ │    │Elasticsearch| |MongoDB│                   │
│ └─────────────────────┘ │    └─────────────┘ └───────┘                   │
│ ┌─────────────────────┐ │    ┌─────┐ ┌───────┐ ┌──────┐ ┌──────┐         │
│ │        Tools        │ │    |Flask| |FastAPI| |Kibana| |Scrapy|         │
│ └─────────────────────┘ │    └─────┘ └───────┘ └──────┘ └──────┘         │
│                         │    ┌─────────────────┐ ┌───┐ ┌──────────┐      │
│                         │    |Generative AI/LLM| |Git| |Amazon AWS|      │
│                         │    └─────────────────┘ └───┘ └──────────┘      │
│                         │    ┌─────────┐ ┌────────────┐                  │
│                         │    |AI Agents| |DigitalOcean|                  │
│                         │    └─────────┘ └────────────┘                  │
└─────────────────────────┴────────────────────────────────────────────────┘

Hobbies & Interest

Contact

Email mail@aniketyadav.me
LinkedIn @yadavaniket
Twitter @_aniketyadav
Telegram @monoatomic
Discord @monoatomic