Here's the updated README.md file with the npm commands for the required packages:
# Lighthouse Crawler
This project is a Node.js script that reads URLs from a CSV file, performs Lighthouse audits on each URL, handles different types of authentication, and outputs the results to another CSV file. The results include performance scores and links to HTML reports generated by Lighthouse.
## Features
- Reads URLs from an input CSV file.
- Supports Basic, Digest, and Page login authentication.
- Runs Lighthouse audits for performance, accessibility, best practices, and SEO.
- Outputs results to an output CSV file with hyperlinks to the audited URLs and Lighthouse reports.
- Includes timestamps for each audit.
## Prerequisites
- Node.js (v14 or later)
- npm (Node Package Manager)
## Installation
1. Clone the repository:
```bash
git clone https://github.com/abhishek-bhardwaj/lighthouseRunner.git
cd lighthouse-crawler
```
2. Install the dependencies:
```bash
npm install chrome-launcher csv-parser csv-writer node-fetch puppeteer lighthouse
```
3. Create a `config.json` file in the root directory with the following structure:
```json
{
"authType": "basic", // or "digest" or "page"
"basicAuth": {
"username": "your-username",
"password": "your-password"
},
"pageAuth": {
"loginUrl": "https://example.com/login",
"usernameField": "#username",
"passwordField": "#password",
"submitField": "#submit",
"username": "your-username",
"password": "your-password"
}
}
```
4. Prepare your input CSV file (`Crawler.csv`) in the following format:
```csv
URL
https://example.com
https://another-example.com
```
## Usage
Run the script:
```bash
node index.jsThis will:
- Read URLs from
Crawler.csv. - Perform Lighthouse audits on each URL.
- Generate HTML reports for each URL.
- Write the results, including hyperlinks to the reports, to
LighthouseResults.csv.
The output CSV (LighthouseResults.csv) will have the following structure:
Timestamp,URL,Performance,Accessibility,Best Practices,SEO,Report
2024-06-03T15:31:13.734Z,"https://example.com","90","80","85","70","=HYPERLINK("/path/to/report.html", "/path/to/report.html")"If there is an error processing a URL, the script will log the error and continue processing the remaining URLs. The CSV output will contain 'Error' for the performance metrics of the failed URLs.
This project is licensed under the MIT License. See the LICENSE file for details.
- Fork the repository.
- Create a new branch (
git checkout -b feature-branch). - Make your changes.
- Commit your changes (
git commit -am 'Add some feature'). - Push to the branch (
git push origin feature-branch). - Open a pull request.
For questions or support, please open an issue in the repository.
Place this `README.md` file in the root directory of your project. It includes the npm commands to install the necessary packages and provides an overview of the project's functionality, setup instructions, usage, and other relevant information.