Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Search: add special BPM filters #12072

Merged
merged 18 commits into from
May 8, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
9f87a54
Search: add BpmFilternode (simplified clone of NumericFilterMatcher)
ronso0 Oct 12, 2023
77da3f8
BpmFilterNode: fuzzy matching with ~ prefix (+-6%), exact search with…
ronso0 Oct 12, 2023
b2490ea
Test: add test for BpmFilterNode, use bitrate in NumericFilterNode tests
ronso0 Oct 8, 2023
6e588ec
Preferences > Library: re-group options, add Track Search group
ronso0 Oct 13, 2023
c39003d
Search: set the BPM fuzzy range in Library preferences
ronso0 Oct 11, 2023
0542824
Search related Tracks: use fuzzy BPM range set in Library preferences
ronso0 Oct 11, 2023
ffc5684
DlgPrefLibrary: update fuzzy BPM range when 'rate_range' changed
ronso0 Feb 22, 2024
61d3708
BpmFilterNode: replace , with . so numpad with localized separator ca…
ronso0 Feb 26, 2024
1e30920
BpmFilterNode: use range when BPM string has decimals
ronso0 Feb 26, 2024
77b5391
BpmFilterNode: simplify match case detection with MatchMode enum
ronso0 Mar 30, 2024
57114ec
BpmFilterNode: add Strict modes argument has decimals
ronso0 Apr 2, 2024
ac4b72b
BpmFilterNode: expand NULL search to cover '-' and 0.00 etc.
ronso0 Apr 4, 2024
9f1a2a2
BpmFilterNode: Strict modes require decimals (not just separator)
ronso0 Apr 4, 2024
382017d
BpmFilterNode: refine Strict modes, rename mode names
ronso0 Apr 4, 2024
0387942
BpmFilterNode: adjust tests
ronso0 Apr 4, 2024
865367d
BpmFilterNode: adjust MatchMode::HalveDouble boundaries
daschuer Apr 6, 2024
50617dd
BpmFilterNode: simplify MatchMode::HalveDouble range caluclations
daschuer Apr 7, 2024
8ee131c
BpmFilterNode: Allow a range in case of floor(halfBpm) ) == halfBpm i…
daschuer Apr 7, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions src/defs_urls.h
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,8 @@
MIXXX_MANUAL_URL "/chapters/user_interface.html#using-cue-modes"
#define MIXXX_MANUAL_SYNC_MODES_URL \
MIXXX_MANUAL_URL "/chapters/djing_with_mixxx#sync-lock-with-dynamic-tempo"
#define MIXXX_MANUAL_TRACK_SEARCH_URL \
MIXXX_MANUAL_URL "/chapters/library.html#finding-tracks-search"
#define MIXXX_MANUAL_BEATS_URL \
MIXXX_MANUAL_URL "/chapters/preferences.html#beat-detection"
#define MIXXX_MANUAL_KEY_URL \
Expand Down
5 changes: 5 additions & 0 deletions src/library/library_prefs.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,11 @@ const ConfigKey mixxx::library::prefs::kSearchDebouncingTimeoutMillisConfigKey =
mixxx::library::prefs::kConfigGroup,
QStringLiteral("SearchDebouncingTimeoutMillis")};

const ConfigKey mixxx::library::prefs::kSearchBpmFuzzyRangeConfigKey =
ConfigKey{
mixxx::library::prefs::kConfigGroup,
QStringLiteral("search_bpm_fuzzy_range")};
daschuer marked this conversation as resolved.
Show resolved Hide resolved

const ConfigKey mixxx::library::prefs::kEnableSearchCompletionsConfigKey =
ConfigKey{
mixxx::library::prefs::kConfigGroup,
Expand Down
2 changes: 2 additions & 0 deletions src/library/library_prefs.h
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,8 @@ extern const ConfigKey kTrackDoubleClickActionConfigKey;

extern const ConfigKey kSearchDebouncingTimeoutMillisConfigKey;

extern const ConfigKey kSearchBpmFuzzyRangeConfigKey;

extern const ConfigKey kEnableSearchCompletionsConfigKey;

extern const ConfigKey kEnableSearchHistoryShortcutsConfigKey;
Expand Down
286 changes: 286 additions & 0 deletions src/library/searchquery.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,8 @@

namespace {

constexpr double kLibraryRoundRange = 0.05;

const QRegularExpression kDurationRegex(QStringLiteral("^(\\d+)(m|:)?([0-5]?\\d)?s?$"));

// The ordering of operator alternatives separated by '|' is crucial to avoid incomplete
Expand All @@ -24,6 +26,8 @@ const QRegularExpression kDurationRegex(QStringLiteral("^(\\d+)(m|:)?([0-5]?\\d)
// > are not necessarily greedy.
const QRegularExpression kNumericOperatorRegex(QStringLiteral("^(<=|>=|=|<|>)(.*)$"));

const QRegularExpression kNullRegex(QStringLiteral("^([0.,]+)$"));

QVariant getTrackValueForColumn(const TrackPointer& pTrack, const QString& column) {
if (column == LIBRARYTABLE_ARTIST) {
return pTrack->getArtist();
Expand Down Expand Up @@ -471,6 +475,288 @@ double DurationFilterNode::parse(const QString& arg, bool* ok) {
return 60 * m + s;
}

// static
constexpr double BpmFilterNode::kRelativeRangeDefault;

// static
double BpmFilterNode::s_relativeRange = kRelativeRangeDefault;

// static
void BpmFilterNode::setBpmRelativeRange(double range) {
// range < 0 would yield zero results because m_dRangeLow > m_dRangeHigh
VERIFY_OR_DEBUG_ASSERT(range >= 0) {
return;
}
s_relativeRange = range;
}

namespace {

inline QString rangeSqlString(double lower, double upper) {
// 'BETWEEN' is inclusive, i.e. returns true if lower <= value <= upper
return QStringLiteral("bpm BETWEEN %1 AND %2")
.arg(QString::number(lower),
QString::number(upper));
}

inline QString rangeUpperExclusiveSqlString(double lower, double upper) {
return QStringLiteral("bpm >= %1 AND bpm < %2")
.arg(QString::number(lower),
QString::number(upper));
}

inline std::pair<double, double> rangeFromTrailingDecimal(double bpm) {
// Set up a range if we have decimals. This will include matches
// for which we show rounded values in the library. For example
// 124.0 finds 123.95 - 124.05
// 124.1 finds 124.05 - 124.15
// 124.92 finds 124.915 - 124.925
if (bpm == 0.0) {
return std::pair<double, double>(bpm, bpm);
}
int numDecimals = 1;
double intPart;
double fractPart = std::modf(bpm, &intPart);
if (fractPart != 0.0) {
QString decimals = QString::number(fractPart);
numDecimals = decimals.split('.').at(1).length();
}
double roundRange = 5 / pow(10, numDecimals + 1);
// Don't search for negative BPM
double lower = std::max(0.0, bpm - roundRange);
double upper = bpm + roundRange;
return std::pair<double, double>(lower, upper);
}

} // namespace

BpmFilterNode::BpmFilterNode(QString& argument, bool fuzzy, bool negate)
: m_matchMode(MatchMode::Invalid),
m_operator("="),
m_bpm(0.0),
m_rangeLower(0.0),
m_rangeUpper(0.0),
m_bpmHalfLower(0.0),
m_bpmHalfUpper(0.0),
m_bpmDoubleLower(0.0),
m_bpmDoubleUpper(0.0) {
QRegularExpressionMatch nullMatch = kNullRegex.match(argument);
if (argument == kMissingFieldSearchTerm || // explicit empty
argument == "-" || // displayed in the BPM column
nullMatch.hasMatch()) { // displayed in the BPM widgets
m_matchMode = MatchMode::Null;
return;
}

QRegularExpressionMatch opMatch = kNumericOperatorRegex.match(argument);
if (opMatch.hasMatch()) {
if (fuzzy) {
// fuzzy can't be combined with operators
// m_matchMode is already Invalid.
return;
}
m_operator = opMatch.captured(1);
argument = opMatch.captured(2);
}

// Replace the locale's decimal separator with .
// This is handy if numbers are typed with the numpad.
argument.replace(',', '.');
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should simply use QLocale().toDouble() below.
This might fail for 123.456,00 in case users have mixed locales (see #13051).
Currently it would fail if users type gb/us thousands separators 1,230.5

Both is acceptable IMO ;)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I considered your solution as quite smart, because we display the bpm with dot separator in the GUI and Library table but use a comma separator in editors. It is unlikely that one is using bpms above 999.

This issue is however only present in case the user has a locale with comma separators. In case of a dot separator local everything is consistent and the hack is probably not that smart. We should make it conditional then.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, ideally we'd have a regex using QLocale::groupSeparator() and QLocale::decimalPoint()

I'd prefer to merge this now to collect alpha/beat feedback and work on the regex in a follow-up.

@daschuer @ywwg Ready to roll?

bool isDouble = false;
double bpm = argument.toDouble(&isDouble);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

bpm is only used after this mode conditional branch, can you move it down to, say, line 600 or so? That way it's closer to where it's being used

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, we need to know if the argument is a double here already, and doing so with toDouble() gives us the bpm.
We may assign m_bpm = bpm right away at the top of the isDouble branch, regardless whether it'll be used or not (that would save the ::Operator case at the end of this function).

if (isDouble) {
// Check if the arg has a decimal separator (even if no digits after that)
bool strictMatch = argument.split('.', Qt::SkipEmptyParts).length() > 1;
if (fuzzy) {
// fuzzy search +- n%
m_matchMode = MatchMode::Fuzzy;
} else if (!opMatch.hasMatch() && !negate) {
// Simple 'bpm:NNN' search.
// Also searches for half/double matches (rounded up/down)
// Center value is turned into range in order to ...
if (strictMatch) {
// find rounded values (hiddendecimals in tracks table / BPM widgets)
m_matchMode = MatchMode::HalveDoubleStrict;
} else {
// find all values with same int base
m_matchMode = MatchMode::HalveDouble;
}
} else {
// Operator or basic query, optional negate
if (m_operator == '=') {
// Explicit search.
// Same center range as HalveDouble/~Strict
// TODO What about -bpm: ? ExplicitNot
if (strictMatch) {
m_matchMode = MatchMode::ExplicitStrict;
} else {
m_matchMode = MatchMode::Explicit;
}
} else {
m_matchMode = MatchMode::Operator;
}
}
} else {
if (fuzzy) {
// fuzzy can't be combined with range
return;
}
// Test if this is a valid range query
QStringList rangeArgs = argument.split("-");
if (rangeArgs.length() == 2) {
bool lowOk = false;
bool highOk = false;
double rangeLower = rangeArgs[0].toDouble(&lowOk);
double rangeUpper = rangeArgs[1].toDouble(&highOk);

if (lowOk && highOk && rangeLower <= rangeUpper) {
// Assign values now to avoid moving bounds out of scope
m_matchMode = MatchMode::Range;
m_rangeLower = rangeLower;
m_rangeUpper = rangeUpper;
return;
}
}
// m_matchMode is already Invalid.
return;
}

// Build/prepare match functions
switch (m_matchMode) {
case MatchMode::Explicit: {
// No decimals: 114 finds 113.95 - 114.99999
m_rangeLower = rangeFromTrailingDecimal(bpm).first;
m_rangeUpper = floor(bpm + 1);
break;
}
case MatchMode::ExplicitStrict: {
// Decimals: 114.0 finds 113.95 - 114.05
std::tie(m_rangeLower, m_rangeUpper) = rangeFromTrailingDecimal(bpm);
break;
}
case MatchMode::Fuzzy: {
m_rangeLower = floor((1 - s_relativeRange) * bpm);
m_rangeUpper = ceil((1 + s_relativeRange) * bpm);
break;
}
case MatchMode::HalveDouble: {
DEBUG_ASSERT(bpm == floor(bpm));
// For instance bpm = 127
// We find everything that is displayed as 127.** in the library
m_rangeLower = bpm - kLibraryRoundRange; // [126.95
m_rangeUpper = bpm + 1; // ]128

double halfBpm = bpm / 2;
if (floor(halfBpm) != halfBpm) {
// In case bpm / 2 is a fractional, we include the *.0 value neighbours
// since fractional bpm values are less common in some genres.
m_bpmHalfLower = floor(halfBpm) - kLibraryRoundRange; // [62.95
} else {
// Here we are for instance with 126
// We find everything that is displayed as 63.** in the library
m_bpmHalfLower = halfBpm - kLibraryRoundRange; // [62.95
}
m_bpmHalfUpper = halfBpm + 1; // ]64

// We find everything which would be found when editing the beat grid via / 2
m_bpmDoubleLower = (bpm * 2) - kLibraryRoundRange; // [253.95
m_bpmDoubleUpper = m_rangeUpper * 2; // ]256
break;
}
case MatchMode::HalveDoubleStrict: { // 57.0
std::tie(m_rangeLower, m_rangeUpper) =
rangeFromTrailingDecimal(bpm); // 56.95 - 57.05
std::tie(m_bpmHalfLower, m_bpmHalfUpper) =
rangeFromTrailingDecimal(bpm / 2); // 28.45 - 28.55
std::tie(m_bpmDoubleLower, m_bpmDoubleUpper) =
rangeFromTrailingDecimal(bpm * 2); // 113.95 - 114.05
break;
}
case MatchMode::Operator: {
m_bpm = bpm;
break;
}
default:
return;
}
}

bool BpmFilterNode::match(const TrackPointer& pTrack) const {
double value = pTrack->getBpm();

switch (m_matchMode) {
case MatchMode::Null: {
return value == 0.0;
}
case MatchMode::Explicit: {
return value >= m_rangeLower && value < m_rangeUpper;
}
case MatchMode::ExplicitStrict:
case MatchMode::Fuzzy:
case MatchMode::Range: {
return value >= m_rangeLower && value <= m_rangeUpper;
}
case MatchMode::HalveDouble: {
return (value >= m_rangeLower && value <= m_rangeUpper) ||
(value >= m_bpmHalfLower && value <= m_bpmHalfUpper) ||
(value >= m_bpmDoubleLower && value <= m_bpmDoubleUpper);
}
case MatchMode::HalveDoubleStrict: {
return (value >= m_rangeLower && value < m_rangeUpper) ||
(value >= m_bpmHalfLower && value <= m_bpmHalfUpper) ||
(value >= m_bpmDoubleLower && value <= m_bpmDoubleUpper);
}
case MatchMode::Operator: {
return (m_operator == "=" && value == m_bpm) ||
(m_operator == "<" && value < m_bpm) ||
(m_operator == ">" && value > m_bpm) ||
(m_operator == "<=" && value <= m_bpm) ||
(m_operator == ">=" && value >= m_bpm);
}
default: // e.g. MatchMode::Invalid
// Show no results to indicate the query is invalid.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this can be a debug assert -- in a release build we can just silently fail, but during development we'd want to know it was happening

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MatchMode::Invalid is not for programmming errors but for malformed typed queries (user error), for example
fuzzy range ~bpm:123-125
fuzzy operator ~bpm:<=95

// Is this good UX?
return false;
}
}

QString BpmFilterNode::toSql() const {
switch (m_matchMode) {
case MatchMode::Null: {
return QString("bpm IS 0");
}
case MatchMode::Explicit: {
return QStringLiteral("bpm >= %1 AND bpm < %2")
.arg(QString::number(m_rangeLower),
QString::number(m_rangeUpper));
}
case MatchMode::ExplicitStrict:
case MatchMode::Fuzzy:
case MatchMode::Range: {
return rangeSqlString(m_rangeLower, m_rangeUpper);
}
case MatchMode::HalveDouble: {
QStringList searchClauses;
searchClauses << rangeUpperExclusiveSqlString(m_rangeLower, m_rangeUpper);
searchClauses << rangeUpperExclusiveSqlString(m_bpmHalfLower, m_bpmHalfUpper);
searchClauses << rangeUpperExclusiveSqlString(m_bpmDoubleLower, m_bpmDoubleUpper);
return concatSqlClauses(searchClauses, "OR");
}
case MatchMode::HalveDoubleStrict: {
QStringList searchClauses;
searchClauses << rangeSqlString(m_rangeLower, m_rangeUpper);
searchClauses << rangeSqlString(m_bpmHalfLower, m_bpmHalfUpper);
searchClauses << rangeSqlString(m_bpmDoubleLower, m_bpmDoubleUpper);
return concatSqlClauses(searchClauses, "OR");
}
case MatchMode::Operator: {
return QString("bpm %1 %2").arg(m_operator, QString::number(m_bpm));
}
default: // MatchMode::Invalid
return QString("bpm IS NULL");
}
}

KeyFilterNode::KeyFilterNode(mixxx::track::io::key::ChromaticKey key,
bool fuzzy) {
if (fuzzy) {
Expand Down
48 changes: 48 additions & 0 deletions src/library/searchquery.h
Original file line number Diff line number Diff line change
Expand Up @@ -184,6 +184,54 @@ class DurationFilterNode : public NumericFilterNode {
double parse(const QString& arg, bool* ok) override;
};

// BPM filter that supports fuzzy matching via ~ prefix.
// If no operator is provided (bpm:123) it also finds half & double BPM matches.
// Half/double values aren't integers, int ranges are used. E.g. bpm:123.1 finds
// 61-61, 123.1 and 246-247 BPM
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// 61-61, 123.1 and 246-247 BPM
// 61-62, 123.1 and 246-247 BPM

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why it finds only a full match in the 1x case but a 1 BPM rang in other cases? I don't think that the underlying use case is different.
Did you consider to always match integer rounded values?

Or distinguish 123.0 for only 123.0 but 123 for 122.5 .. 123.5

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's only a 1 BPM for half and double if the search term has decimals, because it that case it seems unlikely the user has/wants exact half/double matches.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes I agree but why do the user want exact 1x matches but not exact 2x and 0.5x matched.
You can easily have the same track annotate as 123 or 62,5. So it will be found if it is in the 0.5x range but not in the 1x range. That doe not feel reasonable.

Did you consider to use this:

  • 123 find 122.5 .. 123.5 | 62 .. 63 | 245,5 .. 246,5
  • 123.3 find 123.25 .. 123.35 | 62,1 .. 63,2 | 246,55 .. 246,65
    That means that the omitted decimal places are matching to the rounded value of the searched tracks.

Copy link
Member

@daschuer daschuer Apr 4, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The source code comment is now outdated.

class BpmFilterNode : public QueryNode {
public:
static constexpr double kRelativeRangeDefault = 0.06;
static void setBpmRelativeRange(double range);

BpmFilterNode(QString& argument, bool fuzzy, bool negate = false);

enum class MatchMode {
Invalid,
Null, // bpm:- | bpm:000.0 | bpm:0,0 | bpm:""
Explicit, // bpm:=120
ExplicitStrict, // bpm:=120.0
Fuzzy, // ~bpm:120
Range, // bpm:120-130
HalveDouble, // bpm:120
HalveDoubleStrict, // bpm:120.0
Operator, // bpm:<=120
};

// Allows WSearchRelatedTracksMenu to construct the QAction title
std::pair<double, double> getBpmRange() const {
return std::pair<double, double>(m_rangeLower, m_rangeUpper);
}

QString toSql() const override;

private:
bool match(const TrackPointer& pTrack) const override;

MatchMode m_matchMode;

QString m_operator;

double m_bpm;
double m_rangeLower;
double m_rangeUpper;
double m_bpmHalfLower;
double m_bpmHalfUpper;
double m_bpmDoubleLower;
double m_bpmDoubleUpper;

static double s_relativeRange;
};

class KeyFilterNode : public QueryNode {
public:
KeyFilterNode(mixxx::track::io::key::ChromaticKey key, bool fuzzy);
Expand Down
Loading