Add a new backend filter

To add a new filter to the frontend, see this doc.

Design Doc for filters is in Box here. Note that URL format may have changed since it was written.

There are a couple steps necessary to add a new filter. They differ slightly if the filter value can be looked up in the database directly (a “generic filter”), or if it will need to be transformed into a new value when adding it to the query (a “custom filter”).

Filters Background

Filters are represented as an array of ListingsFilterParams, each of which has a $comparison to use for that filter, the filter field, and whether the filter should include listings that are missing data ($include_nulls). This is read from the URL by the listings controller an passed through to the listings service, which then calls addFilters() to actually add the filters to the query.

The query is performed in two steps. An initial innerFilteredQuery gets the listing ids for the listings that match thee filter, sort, and pagination criteria, and then an outer query joins on all the tables necessary to get all the data for a listing.

Preventing SQL Injection Vulnerabilities

For either generic or custom filters, make sure you do not introduce a SQL injections vulnerability. SQL Injection vulnerabilities happen when user provided text is inserted directly into a query, without escaping or validating it. For example:
For a backend URL that filters on the name like ?filter[0][name]=; DROP TABLE Students&limit=10, we get a filters object that looks like

[{
    $comparison: "=",
    name: "; DROP TABLE Students",
}]

If we then build the query with a line like queryBuilder.andWhere(`listing.name = ${filter[0].name}`); we’ve added a SQL injections vulnerability because we’ve inserted the user provided text directly into the query. Our query will look like AND WHERE listing.name = ; DROP TABLE Students

A carefully crafted input to the filter could expose data to an attacker or corrupt the database.

Instead, we need to either

• Validate that the provided text is one of a limited whitelist of allowed options, the way we do with comparisons.
• Use TypeORM’s built in string escaping, as detailed here.

Generic Filter

A generic filter is one where the value provided to the filter is what will be looked up in the database. For example, with ?filter[0][bedrooms]=3, the backend query will include something like WHERE unit_type.numBedrooms = 3.

To add a new generic filter to the backend we:

1. Update the ListingFilterKeys enum with the name of the new filter.
2. Add a property to ListingFilterParams in listing.dto.ts with the new filter key.
3. Add a line to filterTypeToFieldMap in listing.dto.ts with a map to the database field this filter will be referencing.
4. Update the innerFilteredQuery in listings.service.ts to join on the table we need for the new filter, if it’s not on the Listing entity itself.
5. If you’re using one of the comparisons already in the Compare enum in filter.dto.ts, then skip to step 8.
6. If you’re not using one of the comparisons in the Compare enum, then add it to the Compare enum in filter.dto.ts.
7. Then, add a case for your new comparison in the switch statement in addFilters() in backend/core/src/shared/filter/index.ts.
8. Add your new filter to the switch statement in getComparisonForFilter() in ui-components/src/helpers/filters.ts, so the frontend knows which comparison to tell the backend to use. Generally, the rest of the frontend makes assumptions about what comparison to use for a filter (minRent will use >= for example) but the backend needs to be told what comparison to use, so this map is where that comparison is looked up. Eventually this map may get moved to the backend directly.
9. Add tests for your new filter in listings.e2e-spec.ts and listings.service.spec.ts.

Custom Filter

A custom filter is one where the value provided to the filter may not match the value that needs to be looked up in the database. For example, the filter name may be seniorHousing and takes a boolean, but in the database we actually need to check if the value in the reservedCommunityType table is "senior". We can’t just insert "senior" directly into the query.

The steps for adding a custom filter are the same as adding a generic filter, with a few exceptions:

• You’ll probably use the NA comparison type. Every filter must have a comparison type set, but your custom logic probably doesn’t need to be flexible enough to allow arbitrary comparisons, so you can likely hardcode it.
• In addFilters() in backend/core/src/shared/filter/index.ts, add a check for the filter key here, before it drops into the switch statement that switches on the comparison. If it matches your filter type, then call your own code that adds the type of WHERE clause you need.