README.md

Scraper.Net

What is Scraper.Net?

Scraper.Net is a multi-platform, asynchronous .NET library responsible of providing easy post scraping over a variety of sources.

Supported features

• Scraping Facebook pages
• Scraping RSS feeds
• Scraping Twitter users
• Scraping Youtube channels
• Downloading high-quality videos via youtube-dl
• Screenshotting pages via HtmlCssToImage
• Communication with remote scraper nodes via MassTransit
• Persisting posts (via MongoDB) and listening for new posts events

Where to begin?

Refer to the documentation site which contains guides along with an API reference.

Simple Usage

This example is taken from the Getting started page in the documentation site.

For background explanation and breakdown of the code refer to the page.

var provider = new ServiceCollection()
    .AddLogging()
    .AddScraper(builder => builder.AddFacebook())
    .BuildServiceProvider();

var service = provider.GetRequiredService<IScraperService>();

var posts = service.GetPostsAsync("NaftaliBennett", "facebook");

await foreach (var post in posts)
{
    Console.WriteLine(post.Content);
}