Scraping from Arabic news #2016
Conversation
|
Very nice! Do we want to use these accidents for our reports and infographics?
|
* Historic data scraping added (2015-2022)
* Historic data scraping added (2015-2022)
| abstract: str | ||
| title: str | ||
|
|
||
| def __init__(self, _article_pub_date, _abstract, _title): |
There was a problem hiding this comment.
Hi, I would like us to use type hints in function parameters and variables.
| for groupNum in range(0, len(match.groups())): | ||
| groupNum = groupNum + 1 | ||
| f1.write(match.group(groupNum)) |
There was a problem hiding this comment.
Hi,
- In python we use lowercase and underscore for variables and function/method names.
- I prefer not to change the variable loop in the loop. It is better to use a different variable.
| response = requests.get(url) | ||
| print(response.status_code) | ||
|
|
||
| api_key = "AIzaSyD_B16MmHv7mfQNKSanibF_S2ofJgI6Pc0" |
There was a problem hiding this comment.
Are we OK that the API key is in our code, in a public repo?
There was a problem hiding this comment.
Still working on this one..
Accidently pushed it to the PR
| @@ -0,0 +1,9 @@ | |||
| import requests | |||
|
|
|||
There was a problem hiding this comment.
Hi, the style we use for file names is lowercase with underscore between words.
|
@ziv17 Thank you for your comments. Will amend those issues soon. regarding your question, incorporating the obtained data in the database will be carried out after the newsflash been translated (using Google API) and then will undergo the same process as your mainstream data. |
Next steps: