Repost detection through image hashing

[BakkerHenk]

In order to prevent blatant reposting by users or bots, perhaps implement some kind of hashing function that can detect if an image is an exact copy of an image that was posted earlier. It would not have to refuse a posted image, but could be usefull to send warnings to moderators if certain hash values appear more than a few times in a short period of time.

[JustMaier]

Cool idea. We do have perceptual hashing in place so that we can prevent the re-posting of content that was removed as a ToS violation. I suppose we could do something similar for all posts, but I'm not sure how well it would scale since we already have 50M images and checking against all of them would be incredibly slow.

[BakkerHenk]

You're right scaling would be an issue at those numbers. Limiting it to a time window would probably still be an insane amount, but it might help. Workers could run the queries and log the outliers and clean up or something.

[JustMaier]

Yeah, filtering the immediate check to a 3-7 day window could help. Then we could have a job that runs each day to check images against a larger window.