Code

Big bad merge with later cleanup#7

Code

Merged

dobraczka merged 139 commits intomaindobraczka/klinker:mainfrom devdobraczka/klinker:devCopy head branch name to clipboardJul 9, 2024

Commits

Commits on Oct 25, 2023

• Change perturbation parameter in experiment

  dobraczka

  committed
  0b4e79f

  Copy full SHA for 0b4e79f

• Undo perturbation parameter change in experiment

  dobraczka

  committed
  2b7ac96

  Copy full SHA for 2b7ac96

• Adapt learning-rate parameter

  dobraczka

  committed
  6f99c62

  Copy full SHA for 6f99c62

• Enhance GCNFrameEncoder

  dobraczka

  committed
  5083f9e

  Copy full SHA for 5083f9e

• Add defaultfaiss config

  dobraczka

  committed
  04fcc90

  Copy full SHA for 04fcc90

• Merge branch 'main' into dev

  dobraczka

  committed
  0cebead

  Copy full SHA for 0cebead

• Merge branch 'defaultfaiss' into dev

  dobraczka

  committed
  6c09d1b

  Copy full SHA for 6c09d1b

• Add faiss default as default

  dobraczka

  committed
  b3aaee2

  Copy full SHA for b3aaee2

• Add faiss default as default

  dobraczka

  committed
  d752916

  Copy full SHA for d752916

• Set fasttext as default

  dobraczka

  committed
  7786830

  Copy full SHA for 7786830

• Better incorporate force argument in experiment.py

  dobraczka

  committed
  b47a8b8

  Copy full SHA for b47a8b8

• Fix problem in force argument

  dobraczka

  committed
  c4c1b93

  Copy full SHA for c4c1b93

Commits on Oct 26, 2023

• Adapted RelationalDeepBlocker encoding saving

  dobraczka

  committed
  f6f70aa

  Copy full SHA for f6f70aa

• Add additional params to relational deepblocker experiment

  dobraczka

  committed
  a16fd6d

  Copy full SHA for a16fd6d

• Fix learning-rate parameter in relational-deepblocker experiment

  dobraczka

  committed
  37235c6

  Copy full SHA for 37235c6

Commits on Nov 1, 2023

• Updated Architecture diagram

  dobraczka

  committed
  9a0573d

  Copy full SHA for 9a0573d

Commits on Nov 2, 2023

• Use gpu by default with faiss

  dobraczka

  committed
  5a61124

  Copy full SHA for 5a61124

• Improved blocker naming for relational deepblocker

  dobraczka

  committed
  2136d9b

  Copy full SHA for 2136d9b

• Remove ipdb

  dobraczka

  committed
  ed4af2a

  Copy full SHA for ed4af2a

Commits on Nov 6, 2023

• Fixed relationaldeepblocker encoding times

  dobraczka

  committed
  e720944

  Copy full SHA for e720944

Commits on Nov 16, 2023

• Improved (composite) NNbased blocking

  dobraczka

  committed
  8bf300b

  Copy full SHA for 8bf300b

Commits on Nov 23, 2023

• Added GCNDeepBlocker and LightEADeepBlocker

  dobraczka

  committed
  26fbfef

  Copy full SHA for 26fbfef

Commits on Nov 26, 2023

• Added SparseSinkhorn

  dobraczka

  committed
  e443959

  Copy full SHA for e443959

• Adapted default bb_kwargs in experiment

  dobraczka

  committed
  9d8ac78

  Copy full SHA for 9d8ac78

• Adapt sinkhorn casting

  dobraczka

  committed
  f3604e1

  Copy full SHA for f3604e1

• Adapt gpu handling in sparse sinkhorn

  dobraczka

  committed
  06c1c62

  Copy full SHA for 06c1c62

• Added n_candidates parameter

  dobraczka

  committed
  9583ae3

  Copy full SHA for 9583ae3

• Fix some variable names in blockbuilder

  dobraczka

  committed
  6ac8ae0

  Copy full SHA for 6ac8ae0

Commits on Nov 27, 2023

• Remove ipdb

  dobraczka

  committed
  34cb476

  Copy full SHA for 34cb476

• Added warning for n_candidates for kiez

  dobraczka

  committed
  b8fe2ea

  Copy full SHA for b8fe2ea

Commits on Nov 29, 2023

• Fixed parquet problem

  dobraczka

  committed
  d53fb74

  Copy full SHA for d53fb74

• Change faiss default

  dobraczka

  committed
  54530a2

  Copy full SHA for 54530a2

Commits on Nov 30, 2023

• Use faiss in SSI

  dobraczka

  committed
  02c4533

  Copy full SHA for 02c4533

• Use kiez in SSI

  dobraczka

  committed
  7b7c0fa

  Copy full SHA for 7b7c0fa

Commits on Dec 4, 2023

• Use same faiss config everywhere

  dobraczka

  committed
  8a561ef

  Copy full SHA for 8a561ef

Commits on Dec 7, 2023

• Enhanced configuration for block_builders in experiment

  dobraczka

  committed
  ffc6f10

  Copy full SHA for ffc6f10

• Print kiez info

  dobraczka

  committed
  08c0da7

  Copy full SHA for 08c0da7

• Add more print

  dobraczka

  committed
  349b8ae

  Copy full SHA for 349b8ae

• Use newer kiez version

  dobraczka

  committed
  b41de46

  Copy full SHA for b41de46

• Reverse kiez fitting

  dobraczka

  committed
  c43ddad

  Copy full SHA for c43ddad

• Investigate kiez problem

  dobraczka

  committed
  678ab3d

  Copy full SHA for 678ab3d

• Add missing dependency

  dobraczka

  committed
  eb5da42

  Copy full SHA for eb5da42

• Remove obsolete reference

  dobraczka

  committed
  12dec05

  Copy full SHA for 12dec05

Commits on Dec 8, 2023

• Change dist

  dobraczka

  committed
  e496855

  Copy full SHA for e496855

Commits on Mar 25, 2024

• Ruffified, adapted to new version of sylloge

  dobraczka

  committed
  b540115

  Copy full SHA for b540115

Commits on Apr 23, 2024

• Fix sylloge issues, adapt for new kiez

  dobraczka

  committed
  0da55ff

  Copy full SHA for 0da55ff

• Adapt experiment parameters

  dobraczka

  committed
  1f6c265

  Copy full SHA for 1f6c265

Commits on Apr 26, 2024

• Simplified experiment

  dobraczka

  committed
  14a1b56

  Copy full SHA for 14a1b56

Commits on Apr 29, 2024

• Fix STBert problems

  dobraczka

  committed
  14abff7

  Copy full SHA for 14abff7

• Use batch_generator with dask for SBert

  dobraczka

  committed
  c45c3c8

  Copy full SHA for c45c3c8

• Fix values error

  dobraczka

  committed
  5f3b43d

  Copy full SHA for 5f3b43d

• Use different strategy for dask

  dobraczka

  committed
  210221d

  Copy full SHA for 210221d

• Debug

  dobraczka

  committed
  9b3e5c6

  Copy full SHA for 9b3e5c6

• Adapt experiment stuff

  dobraczka

  committed
  233aaea

  Copy full SHA for 233aaea

• Move torch tensor to cpu after NN

  dobraczka

  committed
  74917d5

  Copy full SHA for 74917d5

Commits on Apr 30, 2024

• Simplified LightEA

  dobraczka

  committed
  b352a86

  Copy full SHA for b352a86

• Remove ent_dim arg

  dobraczka

  committed
  f1bba55

  Copy full SHA for f1bba55

• Use torhc normalize

  dobraczka

  committed
  28cfe37

  Copy full SHA for 28cfe37

• Compute dask frames before mapping

  dobraczka

  committed
  98ccc15

  Copy full SHA for 98ccc15

• Added PCA downsampling possibility

  dobraczka

  committed
  fde4774

  Copy full SHA for fde4774

• Move tensors to cpu before writing to disk

  dobraczka

  committed
  cd0c94b

  Copy full SHA for cd0c94b

Commits on May 3, 2024

• New relational and adapted faiss

  dobraczka

  committed
  3e51c7f

  Copy full SHA for 3e51c7f

• Use negative int for None

  dobraczka

  committed
  59f1f07

  Copy full SHA for 59f1f07

• Add possibility to install optional experiment deps

  dobraczka

  committed
  1099cda

  Copy full SHA for 1099cda

• Make torch optional in experiment

  dobraczka

  committed
  9b6c717

  Copy full SHA for 9b6c717

• Extra config does not make sense

  dobraczka

  committed
  9165402

  Copy full SHA for 9165402

• Add HNSW option

  dobraczka

  committed
  d8ee0e2

  Copy full SHA for d8ee0e2

• Find memory problem

  dobraczka

  committed
  7fa27b7

  Copy full SHA for 7fa27b7

• Fixed faiss, adde print

  dobraczka

  committed
  f032c43

  Copy full SHA for f032c43

Commits on May 4, 2024

• Fix possible bug in KlinkerTripleDaskFrame

  dobraczka

  committed
  db45095

  Copy full SHA for db45095

• repartition before transform

  dobraczka

  committed
  82fdf19

  Copy full SHA for 82fdf19

• repartition before transform

  dobraczka

  committed
  c85b56b

  Copy full SHA for c85b56b

Commits on May 7, 2024

• Remove print statement

  dobraczka

  committed
  61663ba

  Copy full SHA for 61663ba

• Fixed count_entities bug

  dobraczka

  committed
  09ee0b4

  Copy full SHA for 09ee0b4

• Address missing import

  dobraczka

  committed
  714544d

  Copy full SHA for 714544d

• Fix head problem

  dobraczka

  committed
  f0f4f80

  Copy full SHA for f0f4f80

Commits on May 8, 2024

• Enable slurm cluster option

  dobraczka

  committed
  5a974e5

  Copy full SHA for 5a974e5

• Fix arg

  dobraczka

  committed
  0c73c57

  Copy full SHA for 0c73c57

• Add local directory

  dobraczka

  committed
  e422a7f

  Copy full SHA for e422a7f

• Add missing arg

  dobraczka

  committed
  cc7757a

  Copy full SHA for cc7757a

• Add missing local_dir

  dobraczka

  committed
  b065e02

  Copy full SHA for b065e02

• Remove unnecessary

  dobraczka

  committed
  b4c94ba

  Copy full SHA for b4c94ba

• Change order

  dobraczka

  committed
  23cc3bb

  Copy full SHA for 23cc3bb

• actually provide local_dir path

  dobraczka

  committed
  7cd0795

  Copy full SHA for 7cd0795

• Add partition size to BlockManager read

  dobraczka

  committed
  af75694

  Copy full SHA for af75694

Commits on May 9, 2024

• Use scale down before eval

  dobraczka

  committed
  865f766

  Copy full SHA for 865f766

• Fix topn in deepblocker

  dobraczka

  committed
  adc1e85

  Copy full SHA for adc1e85

• Fix nn problems

  dobraczka

  committed
  349f690

  Copy full SHA for 349f690

• Fix more nn stuff

  dobraczka

  committed
  56d6bbc

  Copy full SHA for 56d6bbc

• Debug old concat

  dobraczka

  committed
  6597912

  Copy full SHA for 6597912

• Add all old concat

  dobraczka

  committed
  b91c569

  Copy full SHA for b91c569

• Further debugging

  dobraczka

  committed
  9823486

  Copy full SHA for 9823486

• Fix concat

  dobraczka

  committed
  f6f1d81

  Copy full SHA for f6f1d81

Commits on May 10, 2024

• Add alternative eval

  dobraczka

  committed
  123c73a

  Copy full SHA for 123c73a

• Improve later eval

  dobraczka

  committed
  62eb225

  Copy full SHA for 62eb225

• Partition size

  dobraczka

  committed
  3c830f7

  Copy full SHA for 3c830f7

• Fix canonical_name_to_ds bug

  dobraczka

  committed
  b8b3d2b

  Copy full SHA for b8b3d2b

• Fix canonical_name_to_ds bug

  dobraczka

  committed
  385446a

  Copy full SHA for 385446a

• Try spark eval

  dobraczka

  committed
  f6fa445

  Copy full SHA for f6fa445

Commits on May 13, 2024

• Change minimal eval

  dobraczka

  committed
  81189c1

  Copy full SHA for 81189c1

• Use several writes

  dobraczka

  committed
  d3569a0

  Copy full SHA for d3569a0

Commits on May 15, 2024

• Use old eval and concat

  dobraczka

  committed
  94e0b88

  Copy full SHA for 94e0b88

Commits on May 29, 2024

• Added composite blockers

  dobraczka

  committed
  4f7ed94

  Copy full SHA for 4f7ed94

• Missing changes

  dobraczka

  committed
  bab702c

  Copy full SHA for bab702c

Commits on Jun 3, 2024

• Move Client import

  dobraczka

  committed
  1e422c0

  Copy full SHA for 1e422c0

• Fix blocker naming

  dobraczka

  committed
  dee0286

  Copy full SHA for dee0286

• Added attribute clustering

  dobraczka

  committed
  bd46c7e

  Copy full SHA for bd46c7e

Commits on Jun 4, 2024

• Added CompositeClustering

  dobraczka

  committed
  d381eec

  Copy full SHA for d381eec

Commits on Jun 5, 2024

• Added composite to experiment

  dobraczka

  committed
  5e96d21

  Copy full SHA for 5e96d21

• Adapted naming

  dobraczka

  committed
  6641019

  Copy full SHA for 6641019

Commits on Jun 6, 2024

• Adapt - and _ in arguments

  dobraczka

  committed
  c7de009

  Copy full SHA for c7de009

• Use different wandb_dir

  dobraczka

  committed
  31d6692

  Copy full SHA for 31d6692

• Add missing hyphen

  dobraczka

  committed
  1bbc3f3

  Copy full SHA for 1bbc3f3

• Adapt WANDB

  dobraczka

  committed
  1b88d68

  Copy full SHA for 1b88d68

• Add missing self

  dobraczka

  committed
  88f32a8

  Copy full SHA for 88f32a8

• New wandb, change pring to log

  dobraczka

  committed
  020a94d

  Copy full SHA for 020a94d

• Fix consoleresulttracker

  dobraczka

  committed
  1c26a69

  Copy full SHA for 1c26a69

• Add logging to attribute clustering

  dobraczka

  committed
  38f3fb8

  Copy full SHA for 38f3fb8

• Enable data sample

  dobraczka

  committed
  f545db4

  Copy full SHA for f545db4

• Adapt ac

  dobraczka

  committed
  3d34219

  Copy full SHA for 3d34219

• More logging

  dobraczka

  committed
  7e46349

  Copy full SHA for 7e46349

• More logging

  dobraczka

  committed
  805eb23

  Copy full SHA for 805eb23

• More logging

  dobraczka

  committed
  7753e8f

  Copy full SHA for 7753e8f

• Even more logging

  dobraczka

  committed
  62c4bdb

  Copy full SHA for 62c4bdb

• Made unique name optional

  dobraczka

  committed
  ac843d3

  Copy full SHA for ac843d3

• More of the log sauce

  dobraczka

  committed
  d2386fb

  Copy full SHA for d2386fb

Commits on Jun 7, 2024

• Remove logging

  dobraczka

  committed
  9b76acc

  Copy full SHA for 9b76acc

Commits on Jun 10, 2024

• Address type problem

  dobraczka

  committed
  3974cc5

  Copy full SHA for 3974cc5

Commits on Jun 12, 2024

• Add possibility to save clustering

  dobraczka

  committed
  1210290

  Copy full SHA for 1210290

Commits on Jun 13, 2024

• Added UMAP dim reduction for SIF

  dobraczka

  committed
  415b05b

  Copy full SHA for 415b05b

Commits on Jun 17, 2024

• Adapt dim reduce flag

  dobraczka

  committed
  27ff40c

  Copy full SHA for 27ff40c

• Reorder import

  dobraczka

  committed
  c51602b

  Copy full SHA for c51602b

• Adapt experiment for attribute

  dobraczka

  committed
  19d8dbf

  Copy full SHA for 19d8dbf

Commits on Jun 18, 2024

• Adapt CompRelDeepBlocker

  dobraczka

  committed
  4437d51

  Copy full SHA for 4437d51

Commits on Jun 19, 2024

• Fix dimensionality problem

  dobraczka

  committed
  b8a7fa9

  Copy full SHA for b8a7fa9

Commits on Jun 24, 2024

• Add dimensionality reduced fasttext

  dobraczka

  committed
  2f6670c

  Copy full SHA for 2f6670c

• Fix dimensionality reduced fasttext

  dobraczka

  committed
  8702442

  Copy full SHA for 8702442

Commits on Jun 25, 2024

• Add 25 dim option

  dobraczka

  committed
  81523a2

  Copy full SHA for 81523a2

Commits on Jul 9, 2024

• Merge branch 'main' of github.com:dobraczka/klinker into dev

  dobraczka

  committed
  a33e2fe

  Copy full SHA for a33e2fe