parallelized processing w/ PDB - works on Mac, not linux

[acschwartz]

tested wrapper_parallel.py with full8puzzle pdb.
input: 500 random 8-puzzles

worked correctly on Mac - no errors.

error on EC2 / Amazon Linux:

XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
************************* ⚠️ E X C E P T I O N ⚠️ *************************
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

🔥 DatabaseError : malformed database schema (PatternCosts_EmptyTileLocation_8) - near "WITHOUT": syntax error

🧶 multiprocessing.pool.RemoteTraceback:
"""
Traceback (most recent call last):
File "/usr/lib64/python3.7/multiprocessing/pool.py", line 121, in worker
result = (True, func(*args, **kwds))
File "/usr/lib64/python3.7/multiprocessing/pool.py", line 44, in mapstar
return list(map(*args))
File "/home/ec2-user/n-puzzle/n-puzzle/process_batchline_multithreaded.py", line 13, in process_batchline_multithreaded
outcome = solver(argsThisRun, parallel=True, silent=False)
File "/home/ec2-user/n-puzzle/n-puzzle/solver.py", line 279, in solver
res = a_star_search(puzzle, goal_state, size, HEURISTIC, TRANSITION_COST, PDB_CONNECTION)
File "/home/ec2-user/n-puzzle/n-puzzle/npuzzle/search.py", line 41, in a_star_search
frontier = {init_state: (0, HEURISTIC(init_state, goal_state, size, PDB_CONNECTION))}
File "/home/ec2-user/n-puzzle/n-puzzle/npuzzle/heuristics.py", line 86, in pdb_Query_full8puzzle
cost = pdb.queryPDB(table, pattern, db_connection)
File "/home/ec2-user/n-puzzle/n-puzzle/npuzzle/pdb/pdb.py", line 20, in queryPDB
cur.execute("SELECT * from %s where pattern = ?"%(tablename), (pattern,))
sqlite3.DatabaseError: malformed database schema (PatternCosts_EmptyTileLocation_8) - near "WITHOUT": syntax error
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "./wrapper_parallel.py", line 755, in
resultsets_list = p.map(partial(process_batchline_multithreaded, argslist=ARGSLIST), batchlines)
File "/usr/lib64/python3.7/multiprocessing/pool.py", line 268, in map
return self._map_async(func, iterable, mapstar, chunksize).get()
File "/usr/lib64/python3.7/multiprocessing/pool.py", line 657, in get
raise self._value
sqlite3.DatabaseError: malformed database schema (PatternCosts_EmptyTileLocation_8) - near "WITHOUT": syntax error

🥞 Stack:
File "./wrapper_parallel.py", line 783, in
printException(exc, lineno())
File "/home/ec2-user/n-puzzle/n-puzzle/formatting/exceptions.py", line 74, in printException
traceback.print_stack()

XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[acschwartz]

how to check sqlite version:
open python interpreter

import sqlite 3
sqlite3.sqlite_version

results:
my Mac: 3.7.17
Amazon Linux: 3.28.0

clue: https://openwritings.net/pg/android/android-without-rowid-only-supported-sqlite-version-382-or-later
(WITHOUT ROWID is the issue - I was using this to save space)

[acschwartz]

how to update sqlite version bundled with python:
pip install pysqlite3

Mac version now up to date. Both wrapper.py and wrapper_parallel.py still run correctly with full8puzzle pdb without throwing errors.
I thought the issue might have been that the DB was created with an older version of sqlite (<3.8), which causes problems with 'WITHOUT ROWID'

SQLite version 3.8.2 (2013-12-06) or later is necessary in order to use a WITHOUT ROWID table. An attempt to open a database that contains one or more WITHOUT ROWID tables using an earlier version of SQLite will result in a "malformed database schema" error.
https://www.sqlite.org/withoutrowid.html

I have no idea why this worked on my machine to begin with, but I will likely have to rebuild the PDBs now that I have upgraded sqlite.