C# Selenium Web Scraper

About

I have developed a web scraping tool with three options for scraping data from different websites. The first option allows the user to enter a search term and retrieve the basic data (link, title, uploader, and number of views) of the five most recently uploaded YouTube videos based on that search term. The second option allows the user to enter a search term and scrape data on the jobsite https://www.ictjob.be/, including the title, company, location, keywords, and link to the details page of the five most recently uploaded jobs that match the search term. The third option allows the user to select a website and data of their choice, based on a search term entered by the user.

Features

• YouTube Video Scraper: Search and retrieve video metadata (title, channel, views)
• ICT Job Scraper: Extract job listings from ictjob.be
• Custom Web Scraper: Flexible scraping for any website

Technologies Used

Built using C# and Selenium WebDriver for browser automation. The application uses WebDriver to navigate websites and extract structured data from HTML pages.

Responsibility

I was responsible for the entire project including designing the application architecture, implementing the scraping logic for different websites, handling anti-scraping measures, and creating the user interface.

Conclusion

This project was a great learning experience in web scraping and browser automation. I learned how to handle dynamic content, manage WebDriver instances, and parse complex HTML structures.