Codex/cloakbrowser browser engines#93
Merged
Merged
Conversation
- Added CloakBrowser as a dependency in the scrape package. - Implemented computeCacheKey function to handle browser runtime options. - Enhanced caching logic to differentiate between Playwright and Puppeteer. - Created CloakBrowserLauncher to manage launching CloakBrowser for both Playwright and Puppeteer. - Added tests for CloakBrowser integration, including local smoke tests and engine factory tests. - Updated ScreenshotTransformer to utilize new public file storage methods. - Refactored engine factories to inject CloakBrowser launchers into Playwright and Puppeteer options.
- Added new environment variables for browser pool management: - ANYCRAWL_KEEP_ALIVE - ANYCRAWL_BROWSER_IDLE_RETIRE_SECS - ANYCRAWL_BROWSER_MAX_PAGES_PER_BROWSER - ANYCRAWL_BROWSER_MAX_OPEN_PAGES_PER_BROWSER - ANYCRAWL_BROWSER_ISOLATE_CONTEXTS - Updated Docker configurations to include new environment variables. - Enhanced the configuration module to parse and apply new browser pool settings. - Introduced tests to validate the new configuration options and their defaults. - Improved the EngineQueueManager to prevent multiple starts of the same engine. - Added functions to resolve scheduled execution times and build idempotency keys for tasks. - Updated SchedulerManager to handle overdue tasks and ensure proper execution scheduling.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
…cker and README - Added CLOAKBROWSER_CACHE_DIR and CLOAKBROWSER_AUTO_UPDATE environment variables in Dockerfiles for both Playwright and Puppeteer. - Updated README to document the usage of CloakBrowser with self-hosted browser engines. - Removed outdated database migration documentation. - Updated package.json to require Node.js 20 or newer. - Fixed supervisord configuration for proper RPC interface.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.