Skip to content

Want to capture the "div" which is created after scrolling the page  #383

@N-pharmarack

Description

@N-pharmarack

Subject of the issue

I want to capture the items from a site, but half of the items are loaded when I scroll the page(link of the page : https://www.91mobiles.com/mobile-price-list-in-india?page=1 ). A new div ( page_scroll_1) is created after I scroll the the page, and the items I want to capture are there in that div.

I have tried ".scrollTo" method from 'x-ray-phantom' but It didn't work

Your environment

  • version of node: 16.13.0
  • version of npm: 8.1.0

Steps to reproduce

const Xray      = require('x-ray');
const puppeteer = require('puppeteer');
var phantom     = require('x-ray-phantom');

var phantom_opts = {
    webSecurity: false,
    images: false,
    weak: false
};

var x = Xray().driver(phantom(phantom_opts, function (nightmare, done) {
    done
        .useragent("Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/38.0.2125.111 Safari/537.36")
        .goto(nightmare.request.req.url)
        .scrollTo(0, 500)
        .wait(3000)
        .scrollTo(0, 1000)
        .wait(3000)
        .scrollTo(0, 1500)
        .wait(3000)
        .scrollTo(0, 2500)
        .wait(3000)
        .scrollTo(0, 3000)
        .wait(3000)
        .scrollTo(0, 3500)
        .wait(3000)
        .scrollTo(0, 4000)
        .wait(3000)
}));


x('https://www.91mobiles.com/mobile-price-list-in-india?page=1', '.finder_snipet_wrap',[{
  mobile_model: '.name',
  Price: '.price',
  Url: '.name@href'
}])
.write('results.json')

Expected behaviour

It should capture all the item with '.finder_snipet_wrap'

Actual behaviour

Capture's only half items, and skips all the other '.finder_snipet_wrap' that is being created after page scrolling

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions