Create Higher Level Abstractions to Extract Simple Field Types

[akshayphilar]

The unified schema has several text fields, some of which are listed below.

1. Weight
2. Unit
3. Volume
4. Rating
5. Date
6. URL
7. Phone Number
8. Email Address
9. Rank

On a given web page, these could be enmeshed in other text such that it would currently require either an re_search function or a lengthier pipeline expression.

+ $36.56 Shipping & Import Fees Deposit to India

Thus not only would it make pipeline expressions more terse, it would also make them robust since we are abstracting away the intricate logic within well-tested shublang functions.

An interesting side effect would be that it would give us the building blocks to construct an AST for implementing #29

WDYT @VMRuiz @ivankivanov @mabelvj @BurnzZ

[BurnzZ]

@akshayphilar, If I understand this correctly, are we to create abstractions on extracting these fields, quite similar to how price and date are for the following?

• https://github.com/scrapinghub/price-parser
• https://github.com/scrapinghub/dateparser

[akshayphilar]

There may be some sort of pre processing that may be required, but yes essentially we will be wrapping shublang functions around such libs whenever they are available.