Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add search results provided by google #10

Open
ianwang opened this issue Aug 13, 2014 · 4 comments
Open

add search results provided by google #10

ianwang opened this issue Aug 13, 2014 · 4 comments

Comments

@ianwang
Copy link

ianwang commented Aug 13, 2014

when you google something about date, time, currency, weather, google will provide the answer for you, like

'what time is it?'
'usd eur'

what do you think about adding this kind of results?

@jprichardson
Copy link
Owner

I think that'd be awesome, I'm just not sure how to do it without really complicating the module. As you're right, Google provides a lot useful answers to questions posed in natural language. But how do you reliably in a non-hacky manner parse results to "what is 2+2", "what time is it in Australia", "define: someword"? I'm not sure that you can. So, it may be best to leave those sorts of queries to the official API. Thoughts?

@ianwang
Copy link
Author

ianwang commented Aug 13, 2014

I didn't try all of the queries, but I guess there is some certain pattern of it (so far I tested .vk_c), and then basically like what you did:

var linkSel = 'h3.r a'
  , descSel = 'div.s'
  , itemSel = 'li.g'
  , nextSel = 'td.b a span';

I'm happy to send PR, if you don't have much time to test this.

@flesler
Copy link
Contributor

flesler commented Feb 29, 2016

+1 !

Instead of links you should return a result object that has a links attribute plus other attributes.

Other things you could expose on this object:

  • The page URL
  • Page number
  • Search term
  • Scrapped Google response
  • Cheerio Object
  • Response HTML
  • The next callback

I think this would simplify the API and make it more extensible without breaking compatibility.

Of course this one change isn't backwards compatible and would require a major version bump.
You could make it backwards-compatible if the callback's length is 3 but that might be too hacky.

If you don't want to implement this scraping, I think you should at least change the response to an object like I describe and provide the cheerio object in it to allow custom scraping.

This was referenced Feb 29, 2016
@flesler
Copy link
Contributor

flesler commented Mar 4, 2016

#38 proposes a response object, the scrapped result on this issue could be added to it. I actually need it, I can do it on my side once cheerio is exposed but I could add it to this module just as well

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants