-
Notifications
You must be signed in to change notification settings - Fork 66
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Compare two fingerprints. #18
Comments
See https://groups.google.com/d/msg/acoustid/Uq_ASjaq3bw/kLreyQgxKmgJ for a general explanation, https://gist.github.com/lalinsky/1132166 for a very simple example and https://bitbucket.org/acoustid/acoustid-server/src/efb787c16ea1a0f6daf38611d12c85376d971b08/postgresql/acoustid_compare.c?at=master#cl-119 for a more complicated example. Feel free to ask more questions on the acoustid mailing list or on IRC in #acoustid on freenode. |
Cool; thanks for the links, @lalinsky. Would it make sense to add this bitwise comparison process to the library for reuse? I'll leave this issue open as a reminder to explore that possibility. |
Thank you, cool links! |
It probably could be added. The problem I had with adding it directly to Chromaprint is that the simple version is not useful for me at AcoustID. I need to handle various exceptional cases, where simple average of the bit differences does not work well enough, and I have the feeling that the more complicated version is too specific to AcoustID (e.g. identifying only full files, not short snippets, etc.). So I always resorted to only posting examples, not an actual reusable code. |
That makes sense; thanks again for explaining. I'll reopen this ticket as a reminder to explore adding something simple based on your example. |
This is what I used in a python library that simply used the executable. I put it here in case anyone wants it. |
I don't think that code does what you expect. What are the inputs to the function? For comparing with fingerprints, you need an list of 32-bit integers. Here you are taking some base64-encoded string, decode it and then work with it one byte at a time. Also, this kind of bit manipulations in Python are very slow. At the very least you should use an 8-bit popcount lookup table, e.g. https://gist.github.com/lalinsky/1132166 |
It does exactly what I expect. Base64 strings were the input, generated with a (custom) executable, not the library. Not optimal but it worked bitwise. Thanks for the link, I'll keep it in mind. |
Oh, so these are not chromaprint fingerprints? In that case, it does not make much sense to comment here. |
They are base64 encoded chromaprints. Produced by an The encoding is irrelevant, the method used to calculate BER is relevant here. |
Well, I clearly can't convince you, but the code you are using is not comparing the audio fingerprints, only their compressed versions. If it works for you, it's just by luck. What you are doing is like taking two similar text files, compressing them with zip and then comparing the zip files. Even though the original text files were similar, the zip files are completely different. |
What? No, I have 2 regular fingerprints which are base64 encoded in a completely separate custom executable. I then run the executable in a python subprocess, and run this on the output. I'm not comparing the raw audio files myself. The custom executable is generating a fingerprint for each song.The chromaprint is generated by this executable. What's not to get? |
I'm not sure why you posted the code example here, because if it's comparing output of your own program, it's not of much use who use standard acoustid/chromaprint tools. |
The algorithm for the bit error rate calculation is relevant here, it can be applied to any program that uses chromaprints -- if you ignore the bit that converts from base64. |
This would be really useful in pyacoustid. |
This result came up in my searching, I thought I'd add where I ended up in case it helps others.
fun chromaDistance(cl0: List<Int>, cl1: List<Int>): Double = cl0.zip(cl1).map { (c0, c1) ->
(c0 xor c1).countOneBits()
}.also {
require(it.isNotEmpty())
}.average() And it felt like matches < 10.0 started to find duplicates. |
I found this Python library which utilizes fpcalc and does a great job: |
AH! Offsets. I forgot offsets. Dangit. Thank you for the link! |
I took a stab at it. Decompressed the fingerprints, as @lalinsky pointed out. I tried to port acoustid_compare.c . As such, I lay no claim to intellectual property to the code below, if it's correct feel free to include it in this lib. import numpy as np
import acoustid
ACOUSTID_MAX_BIT_ERROR = 2
ACOUSTID_MAX_ALIGN_OFFSET = 120
def popcount(x):
return bin(x).count('1')
def match_fingerprints(a, b):
asize = len(a)
bsize = len(b)
numcounts = asize + bsize + 1
counts = np.zeros(numcounts, dtype=int)
for i in range(asize):
jbegin = max(0, i - ACOUSTID_MAX_ALIGN_OFFSET)
jend = min(bsize, i + ACOUSTID_MAX_ALIGN_OFFSET)
for j in range(jbegin, jend):
biterror = popcount(a[i] ^ b[j])
if biterror <= ACOUSTID_MAX_BIT_ERROR:
offset = i - j + bsize
counts[offset] += 1
topcount = counts.max()
return topcount / min(asize, bsize)
def compare_fingerprints(a, b):
import base64
# pad
from acoustid import chromaprint
a=chromaprint.decode_fingerprint(a)[0]
b=chromaprint.decode_fingerprint(b)[0]
a = np.array(a, dtype=np.int32)
b = np.array(b, dtype=np.int32)
return match_fingerprints(a, b)
#use
_, a = acoustid.fingerprint_file(file_path1)
_, b = acoustid.fingerprint_file(file_path2)
compare_fingerprints(a,b) |
It is now implemented in the library, so I think this issue can be closed. |
Another question. Maybe offtopic, but still. I have two identical tracks from different sources. Physically, the tracks are different (possibly encoding compression). I get their fingerprints. They are of course different. Is it possible to get some distance between fingerprints to compare their similarities. If so, in which direction to look? Thank you.
The text was updated successfully, but these errors were encountered: