Skip to content

widiu7omo/simple-scraper

Repository files navigation

##Simple Scraping News How to install

  1. make sure you already install node.js (LTS recommended)
  2. trough your cmd or terminal type this
npm install
or
yarn install

How to use

type `node server.js` trough terminal / cmd
goto localhost:8081 
or 
#DEMO
simple-scraper-news.herokuapp.com
/ root homepage (GET)
/scraping (view scraped data from url already defined) (GET)
/scraping (fetching data from url and save to database) (POST)
/news (view scraped data from database) (GET)

use postman or insomnia to access /scraping with POST method

Screenshot

fetching with dataType JSON Screenshoot

fetching from web Screenshoot #####TODO:

  • Implement Background Queue. (Sparate process to some pieces to avoid H12 error code from heroku server)
  • add Radar Banjarmasin
  • add post param (dynamic scarping)
  • add UI (make better UX)
  • add pagination for fetching more than one page
  • cleaning the code, (index.js), still looking for param
  • add custom config for database
  • add loading animation while fetching data
  • fix error connect to db4free.net

About

Scraping news content with express and sequelize

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published