Skip to content

Releases: website-scraper/node-website-scraper

v1.0.2

29 Apr 19:02
Compare
Choose a tag to compare
  • Get rid of createOutputObject and return Resource as-is

v1.0.1

29 Apr 12:18
Compare
Choose a tag to compare
  • 8c3f1fa - Fix promise bug
  • 8ec9aa7 - Fix bug with recursion when 2 html objects have links to each other

v1.0.0

27 Apr 19:07
Compare
Choose a tag to compare
  • 92eb86f - Change output format: now it returns full tree of assets for each resource
  • a1b347b - Add prettifyUrls feature
  • e7f8b80 - Add urlFilter feature
  • 00443b6 - Add filnameGenerator feature
  • dc4ab93 - Use lodash instead of underscore
  • e88abb2 - Add missing extensions for html and css resources

Breaking changes
Changed output format.
Earlier - flat array of root resources was returned

[ { url: 'http://example.com', filename: 'index.html' } ];

Now - tree of resources

[ { 
  url: 'http://example.com', 
  filename: 'index.html',
  assets: [ // dependencies of index.html
    { 
      url: 'http://example.com/style.css', 
      filename: 'style.css', 
      assets: [ // dependencies of style.css
        { url: 'http://example.com/img-from-styles.png', filename: 'img-from-styles.png', assets: [] },
      ] 
    }
    /* other dependencies of index.html */
  ]
} ];

v0.3.6

21 Apr 21:25
Compare
Choose a tag to compare
  • 9732eb1 Fix similar css urls not updated

v0.3.5

20 Apr 19:23
Compare
Choose a tag to compare
  • bf0a25e Fixed bug with unspecified protocol in resources on https page

v0.3.4

17 Apr 19:59
Compare
Choose a tag to compare
  • 8e85c73 - Fix loading from <img srcset="">

v0.3.3

17 Mar 12:17
Compare
Choose a tag to compare

v0.3.2

21 Feb 12:18
Compare
Choose a tag to compare

v0.3.1

29 Jan 13:27
Compare
Choose a tag to compare
  • 16a28ef update dependencies
  • bef139f add options maxDepth and recursive

v0.3.0

05 Aug 19:55
Compare
Choose a tag to compare
  • 69ab9eb - refactor
  • 9636962 - improve detection of duplicated urls
  • b2d2bed - improve recognizing of resource type
  • remove log from options
  • cover with tests

Breaking changes

  • filename returned by scrape was changed - now it contains relative to directory path
var options = {
  urls: 'http://example.com',
  directory: '/path/to/save'
};
scrape(options).then(console.log); 
// earlier: [ { url: 'http://example.com', filename: '/path/to/save/index.html' } ];
// now:  [ { url: 'http://example.com', filename: 'index.html' } ];