- ApiResponse: add readToBuffer option to defer/stream body instead of eagerly buffering
- TableDataProvider: implement HTML table parser with per-column provider support
- StealthConfig: add 10s page load timeout and copyCookiesFrom parameter for cookie sharing
- StealthUnitDownloader: catch WebDriverTimeoutException on navigation, log warning instead of throwing
- Bump version to 2.9.0
Added JetBrains Rider IDE configuration files and a backup for Beam.Api.csproj. Updated aeqw89.Beam project version to 2.7.0 and package references, including Selenium.WebDriver and System.IO.Hashing. Enhanced RelationalDataProvider to support NextSibling and PreviousSibling relations and configurable traversal distance.
- Added `AnchorDataProvider`, `AnchorCollectionDataProvider`, `ContentsDataProvider`, `ContentsArrayDataProvider`, `DropDownDataProvider`, `ListContentDataProvider`, and `ParagraphedContentDataProvider` for enhanced data extraction flexibility.
- Updated project version to 2.5.0.
- Added `RemoteAddress` property to `StealthConfig` for remote WebDriver support.
- Refactored browser driver creation logic with `DriverDefinition` for enhanced consistency.
- Improved error handling in browser fallback mechanism.
- Incremented project version to 2.4.6.
- Removed obsolete data providers: `AnchorCollectionDataProvider`, `ContentsDataProvider`, and others, consolidating logic into new composable providers.
- Added `ComposeDataProviders`, `SelectDataProvider`, and `RelationalDataProvider` for improved flexibility and reusability.
- Introduced `IManySelectionComposableDataProvider` interface to support multiple-node selection.
- Enhanced `UnitDownloader` with more robust progress tracking.
- Updated package references and project dependencies for consistency.
- Improved error handling in `StealthConfig` initialization for better fallback on browser drivers.
- Incremented project version to 2.4.5.
Replaces specialized binary and HTML downloaders with a generic, options-driven UnitDownloader and UnitFragmentDownloader pattern. Introduces UnitDownloaderOptions and builder classes for flexible configuration, updates interfaces and method signatures to support progress reporting, and removes redundant binary-specific classes. Updates Playwright and Stealth downloaders to use the new generic base, and adds improved error handling and reporting. Also updates dependency versions and makes minor API consistency improvements across the Fluent and Models layers.
- Bumped Microsoft.Extensions.Logging packages to version 9.0.9 across all projects.
- Updated aeqw89.Beam project version to 2.1.4.
- Added new transitive package references, including Microsoft.Recognizers.Text.Number, Microsoft.Playwright, EntityFramework, and others.
- Commented out or removed Beam.Temporary.Cli references.
- Enhanced package structure by rearranging content includes and cleaning up redundant package references.
- Introduced modularity by splitting Beam into new projects: Beam.Abstractions, Beam.Models, and Beam.Downloaders.
- Refactored existing classes into appropriate namespaces and projects.
- Replaced specific implementations with abstractions (e.g., SourceLinkBuilder to LinkBuilder, State to IState, etc.).
- Updated interfaces: added ITemplate, IArticleData, IDownloadReport, and others for improved extensibility.
- Removed deprecated classes like SourceLinkBuilder and StateChangerFactory.
- Enhanced link handling in downloaders by refactoring to use `string` over `SourceLink`.
- Consolidated shared logic under Beam.Abstractions.
Added new Beam.Fluent and Beam.Models projects with staged download builder and data context models. Refactored and moved model classes from Beam.Temporary.Cli to Beam.Models. Added new data providers and extended DataBindings in Beam.Dynamic. Renamed Beam.Puppeteer to Beam.Playwright and updated related classes. Updated project references and package versions. Removed obsolete and unused files from Beam.Temporary.Cli.
Added a static `Empty` method to `DownloadEnumerable` for
creating an empty instance. Updated link handling in
`SequentialDownloader` to use `AbsoluteUri` instead of
`ToString()`, ensuring correct link representation.
These changes improve usability and ensure consistency in
link formatting.
feat: add PuppetConfig and integrate with CLI
Introduced a new PuppetConfig class in the Puppeteer
namespace to manage Puppeteer configurations. Updated
the CLI project to reference the Puppeteer project and
added a new method in DownloadBuilder for using a
Puppet manipulator.
This change enables better configuration management
for Puppeteer within the CLI.
```
Refactor the transformers in the downloader classes to use
ByteDocument instead of byte arrays. This change improves type
safety and clarity in handling document content during
downloads, ensuring that the transformations are more
consistent and maintainable.
Updated project files for `Beam.Dynamic`, `Beam.Exports`, `Beam.Puppeteer`, `Beam.Temporary.Cli`, and `Beam` to include new XML headers, reorganize property groups, and add project references.
Modified `PuppetedUnitDownloader` to support additional parameters for async transformers. Changed return types in `CommonTransformers` to `AsyncTransformer` for asynchronous processing.
Significant refactoring in `DownloadBuilder`, `DownloadContext`, and `DownloadContextBuilder` to introduce generic parameters and improve context management. Updated `SequentialDownloader`, `SequentialFragmentDownloader`, and `UnitDownloader` to accommodate new async transformer types.
Introduced `TypeExtensions` for unique type name generation and added `UnitFragmentDownloaderBinary` for handling binary downloads. Updated solution file to include the new `aeqw89.Beam` project, ensuring proper references across the solution.
These changes enhance the asynchronous capabilities of the Beam library, improve type safety, and streamline the downloading process.
qwsdcvghyu89