-
-
Notifications
You must be signed in to change notification settings - Fork 1.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
tsan fix browsethread #13872
base: 2.5
Are you sure you want to change the base?
tsan fix browsethread #13872
Changes from 1 commit
6ce3d0b
043971e
1aec9f9
7dd247d
2433eae
a286eb4
10d83fa
f39cf96
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -14,80 +14,109 @@ | |
constexpr int kRowBatchSize = 10; | ||
} // namespace | ||
|
||
QWeakPointer<BrowseThread> BrowseThread::m_weakInstanceRef; | ||
static QMutex s_Mutex; | ||
|
||
/* | ||
* This class is a singleton and represents a thread | ||
* that is used to read ID3 metadata | ||
* from a particular folder. | ||
* | ||
* The BrowseTableModel uses this class. | ||
* Note: Don't call getInstance() from places | ||
* Note: Don't call getInstanceRef() from places | ||
* other than the GUI thread. BrowseThreads emit | ||
* signals to BrowseModel objects. It does not | ||
* make sense to use this class in non-GUI threads | ||
*/ | ||
BrowseThread::BrowseThread(QObject *parent) | ||
: QThread(parent) { | ||
m_bStopThread = false; | ||
m_model_observer = nullptr; | ||
//start Thread | ||
BrowseThread::BrowseThread(QObject* parent) | ||
: QThread{parent}, m_model_observer{}, m_bRun{} { | ||
// Start thread | ||
start(QThread::LowPriority); | ||
|
||
qDebug() << "Wait to start browser background thread"; | ||
// Wait until the thread is running | ||
m_mutex.lock(); | ||
while (!m_bRun) { | ||
m_condition.wait(&m_mutex); | ||
} | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I have not understood the loop. Isn't there only one event/codition we are waiting for? m_bRun shall become a semaphore? On the other hand syncronizing the thread start by halting the main thread feels wrong. It would be better to protect the shared sources explicit. What are the shared resources in this case we have tsan complains? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yes, we are waiting for m_bRun to be set to true, as set by the thread::run method. A semaphore would work as well, but since we already have the condition, I take advantage of it. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. It is not very clear to me what this problem is exactly, but this is the tsan report: WARNING: ThreadSanitizer: data race (pid=48840)
Read of size 8 at 0x0001466d5f28 by thread T29:
#0 void doActivate<false>(QObject*, int, void**) <null>:230508612 (mixxx:arm64+0x10011f4e8)
#1 QMetaObject::activate(QObject*, QMetaObject const*, int, void**) <null>:230508612 (mixxx:arm64+0x10011db88)
#2 QThread::started(QThread::QPrivateSignal) <null>:230508612 (mixxx:arm64+0x1001e8c5c)
#3 QThreadPrivate::start(void*) <null>:230508612 (mixxx:arm64+0x100248f38)
Previous write of size 8 at 0x0001466d5f28 by main thread (mutexes: write M0, write M1, write M2):
#0 QObjectPrivate::addConnection(int, QObjectPrivate::Connection*) <null>:230508612 (mixxx:arm64+0x10011bdac)
#1 QObjectPrivate::connectImpl(QObject const*, int, QObject const*, void**, QtPrivate::QSlotObjectBase*, int, int const*, QMetaObject const*) <null>:230508612 (mixxx:arm64+0x100121dc4)
#2 QObject::connectImpl(QObject const*, void**, QObject const*, void**, QtPrivate::QSlotObjectBase*, Qt::ConnectionType, int const*, QMetaObject const*) <null>:230508612 (mixxx:arm64+0x100121790)
#3 QMetaObject::Connection QObject::connect<void (BrowseThread::*)(BrowseTableModel*), void (BrowseTableModel::*)(BrowseTableModel*)>(QtPrivate::FunctionPointer<void (BrowseThread::*)(BrowseTableModel*)>::Object const*, void (BrowseThread::*)(BrowseTableModel*), QtPrivate::FunctionPointer<void (BrowseTableModel::*)(BrowseTableModel*)>::Object const*, void (BrowseTableModel::*)(BrowseTableModel*), Qt::ConnectionType) qobject.h:223 (mixxx:arm64+0x100b046e4)
#4 BrowseTableModel::BrowseTableModel(QObject*, TrackCollectionManager*, RecordingManager*) browsetablemodel.cpp:163 (mixxx:arm64+0x100b05824) The code in the main thread: 162 m_pBrowseThread = BrowseThread::getInstanceRef();
163 connect(m_pBrowseThread.data(),
164 &BrowseThread::clearModel,
165 this,
166 &BrowseTableModel::slotClear,
167 Qt::QueuedConnection); So, my conclusion was that the connection to the thread before it was running was problematic. I will re-assess that this is indeed the case, but I do think that is it. And if so, I don't see a problem with waiting for the thread to have started. That happens only once, and it will be fast anyway. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
This reusing makees the code harder to read. So I would prefer two primitived for two tasks. The ThreadSanitizer warning is really confusing, what does it check for? Whether a variable is written/read from two threads? It must be a control structure in Qt. However waiting for the thread to start should not change the issue for my undertsanding. But if it is the solution i don't mind. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. How does the solution work? I am afraid instead of a solution, we trick ThreadSanitizer. Is there a way to find out what 0x0001466d5f28 is and which mutexted I are considered? Or is it s QT bug? Is 2.4 with Qt 5 affected as well. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Anyway, I am now building Qt5. But it is on purpose that the 2.5 branch of mixxx vcpkg has Qt6 as the default? I am confused now as to what we are supposed to use for Mixxx 2.5. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I am running into lots of problems trying to build against Qt5, so it would be much appreciated if you could help me building Qt6 with vcpkg with this Qt sanitize thread option. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Mixxx 2.4.2 will be our last Qt5 release. So there is IMHO no need to deal with QT5 in this case. We just keep the Qt5 support in our 2.5 branch just in case we want to test regresions. I suggest to issue a PR against our vcpkg branch and use the build package downloaded from GitHub. For my undersanding you jsut need to add the sanitizer flag at the position I pointed out above. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yes, I prefer to use Qt 6, but you pointed it out in the qt5 portfile.cmake. The qt6 portfile.cmake is completely different, and I don't see where to add this option. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. It looks like this works. I'd prefer to set this from my triplets file arm64-osx-tsan.cmake, but for now this have to do. diff --git a/ports/qtbase/portfile.cmake b/ports/qtbase/portfile.cmake list(APPEND FEATURE_OPTIONS -DCMAKE_DISABLE_FIND_PACKAGE_Libudev:BOOL=ON) |
||
m_mutex.unlock(); | ||
qDebug() << "Browser background thread started"; | ||
} | ||
|
||
BrowseThread::~BrowseThread() { | ||
qDebug() << "Wait to finish browser background thread"; | ||
m_bStopThread = true; | ||
//wake up thread since it might wait for user input | ||
m_locationUpdated.wakeAll(); | ||
//Wait until thread terminated | ||
//terminate(); | ||
qDebug() << "Wait to terminate browser background thread"; | ||
// Inform the thread we want it to terminate | ||
m_mutex.lock(); | ||
m_bRun = false; | ||
m_condition.wakeOne(); | ||
m_mutex.unlock(); | ||
// Wait until thread terminated | ||
wait(); | ||
qDebug() << "Browser background thread terminated!"; | ||
qDebug() << "Browser background thread terminated"; | ||
} | ||
|
||
// static | ||
BrowseThreadPointer BrowseThread::getInstanceRef() { | ||
BrowseThreadPointer strong = m_weakInstanceRef.toStrongRef(); | ||
// We create a single BrowseThread instead which is used by multiple | ||
// BrowseTableModel instances. We store a weakpointer so when all | ||
// BrowseTableModel have been destroyed, so it the BrowseThread. | ||
|
||
static QWeakPointer<BrowseThread> s_weakInstanceRef; | ||
static QMutex s_mutex; | ||
|
||
s_mutex.lock(); | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. or maybe this mutex placement. either way, both are improvements. EDIT: no, I reverted this. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Let's understand and double check that. The old code IMHO does not suffere any issues: https://doc.qt.io/qt-6/qsharedpointer.html#thread-safety -> Reference counting is thread safe. We only must protect the There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Ok, I will check again where the issue comes from, here or the thread being used before being started. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. You are right. The original code was correct. I have reverted it, except for making
local variables in the function instead of globals. Please resolve if you are okay with the latest version. |
||
BrowseThreadPointer strong = s_weakInstanceRef.toStrongRef(); | ||
if (!strong) { | ||
s_Mutex.lock(); | ||
strong = m_weakInstanceRef.toStrongRef(); | ||
if (!strong) { | ||
strong = BrowseThreadPointer(new BrowseThread()); | ||
m_weakInstanceRef = strong.toWeakRef(); | ||
} | ||
s_Mutex.unlock(); | ||
strong = BrowseThreadPointer(new BrowseThread()); | ||
s_weakInstanceRef = strong.toWeakRef(); | ||
} | ||
s_mutex.unlock(); | ||
|
||
return strong; | ||
} | ||
|
||
void BrowseThread::executePopulation(mixxx::FileAccess path, BrowseTableModel* client) { | ||
m_path_mutex.lock(); | ||
void BrowseThread::requestPopulateModel(mixxx::FileAccess path, BrowseTableModel* client) { | ||
// Inform the thread to populate a new path. | ||
// Note: if another request is currently being processed, this will replace it. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. What is explained in the comment in line 76 seems weird to me. it makes sense when the client is the same, but what if it's a different client? anyway, I didn't want to change the behaviour. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think it works, because only one table is shown at a time. But indeed an area of improvement. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. That makes sense. |
||
|
||
qDebug() << "Request populate model" << path.info() << client; | ||
m_mutex.lock(); | ||
m_path = std::move(path); | ||
m_model_observer = client; | ||
m_path_mutex.unlock(); | ||
m_locationUpdated.wakeAll(); | ||
m_condition.wakeOne(); | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. What can happen if the thread is not yet waiting? In this case no one is waked up. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. if m_path has a value, the thread will not enter the wait state. |
||
m_mutex.unlock(); | ||
} | ||
|
||
void BrowseThread::run() { | ||
QThread::currentThread()->setObjectName("BrowseThread"); | ||
|
||
// Inform the constructor the thread started | ||
m_mutex.lock(); | ||
m_bRun = true; | ||
m_condition.wakeOne(); | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. wakeOne() and wait() below is confusing, Let's use a second sync method. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I put the request in a separate method. |
||
m_mutex.unlock(); | ||
|
||
while (true) { | ||
// Wait for a new population request, or for a termination request | ||
qDebug() << "Wait for a new population request"; | ||
m_mutex.lock(); | ||
while (!m_path.isSet() && m_bRun) { | ||
m_condition.wait(&m_mutex); | ||
} | ||
auto path = std::move(m_path); | ||
auto modelObserver = m_model_observer; | ||
auto bRun = m_bRun; | ||
m_path = mixxx::FileAccess(); | ||
m_mutex.unlock(); | ||
|
||
while (!m_bStopThread) { | ||
//Wait until the user has selected a folder | ||
m_locationUpdated.wait(&m_mutex); | ||
Trace trace("BrowseThread"); | ||
|
||
//Terminate thread if Mixxx closes | ||
if(m_bStopThread) { | ||
break; | ||
if (!bRun) { | ||
qDebug() << "Termination request"; | ||
// Terminate thread if Mixxx closes | ||
return; | ||
} else { | ||
qDebug() << "New population request"; | ||
// Populate the model | ||
populateModel(path, modelObserver); | ||
} | ||
// Populate the model | ||
populateModel(); | ||
} | ||
m_mutex.unlock(); | ||
} | ||
|
||
namespace { | ||
|
@@ -112,29 +141,27 @@ | |
|
||
} // namespace | ||
|
||
void BrowseThread::populateModel() { | ||
m_path_mutex.lock(); | ||
auto thisPath = m_path; | ||
BrowseTableModel* thisModelObserver = m_model_observer; | ||
m_path_mutex.unlock(); | ||
void BrowseThread::populateModel(mixxx::FileAccess path, BrowseTableModel* modelObserver) { | ||
m0dB marked this conversation as resolved.
Show resolved
Hide resolved
|
||
// Executed by the thread run for incoming populate model requests | ||
|
||
if (!thisPath.info().hasLocation()) { | ||
if (!path.info().hasLocation()) { | ||
// Abort if the location is inaccessible or does not exist | ||
qWarning() << "Skipping" << thisPath.info(); | ||
qWarning() << "Skipping" << path.info(); | ||
return; | ||
} | ||
qDebug() << "populateModel" << path.info() << modelObserver; | ||
|
||
// Refresh the name filters in case we loaded new SoundSource plugins. | ||
const QStringList nameFilters = SoundSourceProxy::getSupportedFileNamePatterns(); | ||
|
||
QDirIterator fileIt(thisPath.info().location(), | ||
QDirIterator fileIt(path.info().location(), | ||
nameFilters, | ||
QDir::Files | QDir::NoDotAndDotDot); | ||
|
||
// remove all rows | ||
// This is a blocking operation | ||
// see signal/slot connection in BrowseTableModel | ||
emit clearModel(thisModelObserver); | ||
emit clearModel(modelObserver); | ||
|
||
QList<QList<QStandardItem*>> rows; | ||
rows.reserve(kRowBatchSize); | ||
|
@@ -144,15 +171,19 @@ | |
int row = 0; | ||
// Iterate over the files | ||
while (fileIt.hasNext()) { | ||
// If a user quickly jumps through the folders | ||
// the current task becomes "dirty" | ||
m_path_mutex.lock(); | ||
auto newPath = m_path; | ||
m_path_mutex.unlock(); | ||
|
||
if (thisPath.info() != newPath.info()) { | ||
qDebug() << "Abort populateModel()"; | ||
populateModel(); | ||
m0dB marked this conversation as resolved.
Show resolved
Hide resolved
|
||
// If while processing a new population request or a termination request | ||
// was received, we abort the current request. This happens if a user | ||
// quickly jumps through the folders and the current task becomes | ||
// "dirty". | ||
m_mutex.lock(); | ||
const bool abortProcess = !m_bRun || | ||
(m_path.isSet() && | ||
(m_path.info() != path.info() || | ||
modelObserver != m_modelObserver)); | ||
Check failure on line 182 in src/library/browse/browsethread.cpp GitHub Actions / clazy
Check failure on line 182 in src/library/browse/browsethread.cpp GitHub Actions / clang-tidy
Check failure on line 182 in src/library/browse/browsethread.cpp GitHub Actions / Ubuntu 22.04
Check failure on line 182 in src/library/browse/browsethread.cpp GitHub Actions / coverage
Check failure on line 182 in src/library/browse/browsethread.cpp GitHub Actions / macOS 12 x64
Check failure on line 182 in src/library/browse/browsethread.cpp GitHub Actions / macOS 12 arm64
|
||
m_mutex.unlock(); | ||
if (abortProcess) { | ||
qDebug() << "Abort populateModel"; | ||
// The run function will take care of the new request. | ||
return; | ||
} | ||
|
||
|
@@ -162,7 +193,7 @@ | |
|
||
const auto fileAccess = mixxx::FileAccess( | ||
mixxx::FileInfo(fileIt.next()), | ||
thisPath.token()); | ||
path.token()); | ||
{ | ||
mixxx::TrackMetadata trackMetadata; | ||
// Both resetMissingTagMetadata = false/true have the same effect | ||
|
@@ -302,14 +333,14 @@ | |
// Will limit GUI freezing | ||
if (row % kRowBatchSize == 0) { | ||
// this is a blocking operation | ||
emit rowsAppended(rows, thisModelObserver); | ||
emit rowsAppended(rows, modelObserver); | ||
qDebug() << "Append" << rows.count() << "tracks from " | ||
<< thisPath.info().locationPath(); | ||
<< path.info().locationPath(); | ||
rows.clear(); | ||
} | ||
// Sleep additionally for 20ms which prevents us from GUI freezes | ||
msleep(20); | ||
} | ||
emit rowsAppended(rows, thisModelObserver); | ||
qDebug() << "Append last" << rows.count() << "tracks from" << thisPath.info().locationPath(); | ||
emit rowsAppended(rows, modelObserver); | ||
qDebug() << "Append last" << rows.count() << "tracks from" << path.info().locationPath(); | ||
} |
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -30,7 +30,7 @@ class BrowseThread : public QThread { | |
Q_OBJECT | ||
public: | ||
virtual ~BrowseThread(); | ||
void executePopulation(mixxx::FileAccess path, BrowseTableModel* client); | ||
void requestPopulateModel(mixxx::FileAccess path, BrowseTableModel* client); | ||
void run(); | ||
static BrowseThreadPointer getInstanceRef(); | ||
|
||
|
@@ -41,16 +41,13 @@ class BrowseThread : public QThread { | |
private: | ||
BrowseThread(QObject *parent = 0); | ||
|
||
void populateModel(); | ||
void populateModel(mixxx::FileAccess path, BrowseTableModel* client); | ||
|
||
QMutex m_mutex; | ||
QWaitCondition m_locationUpdated; | ||
volatile bool m_bStopThread; | ||
QWaitCondition m_condition; | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. QWaitConditions are normally named for the condition that shall become true. Just "condition" is a bit poor. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. the thing is, I am using it for 2 things: waiting for the thread to be started and waiting for a new populate request. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I have renamed the member vars for clarity, to express that these are requests (between the main thread and the browse thread) and what. Please review and resolve. |
||
|
||
// You must hold m_path_mutex to touch m_path or m_model_observer | ||
QMutex m_path_mutex; | ||
// You must hold m_mutex to touch m_path, m_model_observer or m_bRun | ||
mixxx::FileAccess m_path; | ||
BrowseTableModel* m_model_observer; | ||
|
||
static QWeakPointer<BrowseThread> m_weakInstanceRef; | ||
bool m_bRun; | ||
}; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this fixes the data race
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Confirmed. This is the fix for the data race. Connecting the signals/slots while the thread is not yet running triggers the issue.