Question: Merge function fails due to the "transform" key not being found

[rob-scholten95]

I have been trying to merge four tiles, downloaded through WFS in Blender:

3DBAG tiles:
['9/248/576', '9/248/580', '9/252/576', '9/252/580']

The code i use for loading in the file to be read:

with open(filename, 'r', encoding='utf-8') as file:
	cityjson_cm = cjio.cityjson.load(filename, True)

Now, in this file, I have verified multiple times that there is no "transform" key in the file. Now when I load it in as an normal JSON,

with open(filename, 'r', encoding='utf-8') as file:
	cityjson_cm = cjio.cityjson.load(filename, True)

the 'transform' key does exist. The key also exists when I download the tile from the WFS service:

try:
	response = requests.get(url)
							
	with open(fname, "wb") as file:
		data = response.content
		file.write(data) 
		fnames.append(fname)
			
	except urllib.error.HTTPError as err:
              print(err)

This is the output:

CityJSON version = 2.0
EPSG = 7415
bbox = [ 75588.586 450907.906 -1.835 76602.266 451895.625 56.600 ]
=== CityObjects ===
|-- Building (1663)
    |-- BuildingPart (1668)
===================
materials = False
textures = False
CityJSON version = 2.0
EPSG = 7415
bbox = [ 75585.469 451852.469 -0.438 76606.789 452902.656 45.454 ]
=== CityObjects ===
|-- Building (3555)
    |-- BuildingPart (3557)
===================
materials = False
textures = False
CityJSON version = 2.0
EPSG = 7415
bbox = [ 76529.047 450881.562 -3.081 77629.922 451934.500 41.518 ]
=== CityObjects ===
|-- Building (1372)
    |-- BuildingPart (1374)
===================
materials = False
textures = False

of this code:

with open(filename, 'r', encoding='utf-8') as file:
			#json_cm = json.load(file)
			#print(json_cm['transform'])
			
			cityjson_cm = cjio.cityjson.load(filename, True)
			print(cityjson_cm)

this is the collapsed structure of the downloaded JSON file, where the transform key is visible:

I am probably missing something, but i am desperate. Any help would be greatly appreciated!

[hugoledoux]

It's loaded but it's not printed by that function.

If you do this you should see it:

print(cityjson_cm.transform)

See there how to use it:
https://cjio.readthedocs.io/en/latest/api_tutorial_basics.html#Geometry-boundaries-and-Semantic-Surfaces

Hope this helps

[rob-scholten95]

Hey!

thanks for your help! I can now see the 'transform' information using the solution you proposed. I still can't get it to work properly when trying to merge the different tiles together using merge().

def merge_city_json(filenames):
	
	cms = []

	cm = cityjson.load(filenames[0], transform=True)
	print(f"Loaded a citymodel with {len(cm.cityobjects)} CityObjects")
		
	for p in filenames[1:]:
		_cm = cityjson.load(p, transform=True)
		cms.append(_cm)
		print(f"Loaded a citymodel with {len(_cm.cityobjects)} CityObjects")
			
	cm.merge(cms)  

it results in this error:

Python: Traceback (most recent call last):
  File "F:\Projects\00_Documents\JSON\JSON import using python4.1.blend\Text", line 247, in <module>
  File "F:\Projects\00_Documents\JSON\JSON import using python4.1.blend\Text", line 245, in main
  File "F:\Projects\00_Documents\JSON\JSON import using python4.1.blend\Text", line 172, in merge_city_json
  File "C:\Users\Rob\AppData\Roaming\Python\Python311\site-packages\cjio\cityjson.py", line 1263, in merge
    imp_digits = math.ceil(abs(math.log(self.j["transform"]["scale"][0], 10)))
                                        ~~~~~~^^^^^^^^^^^^^
KeyError: 'transform'

[rob-scholten95]

Still trying to figure this out. On my work machine it works flawlessly, but on my personal machine it always stops with this error. I have double checked the versions and have made multiple venv's with the requirements.txt from my work PC's venv. Anyone any clue?

[balazsdukai]

Hi @rob-scholten95 , could you try:

def merge_city_json(filenames):
	
	cms = []

	cm = cityjson.load(filenames[0], transform=False)
	print(f"Loaded a citymodel with {len(cm.cityobjects)} CityObjects")
		
	for p in filenames[1:]:
		_cm = cityjson.load(p, transform=True)
		cms.append(_cm)
		print(f"Loaded a citymodel with {len(_cm.cityobjects)} CityObjects")
			
	cm.merge(cms)  

You'll find some more material here: https://github.com/3DGI/3dbag-workshop-foss4gnl2021 . The workshop explains how the different parts of cjio work together at the moment.

[rob-scholten95]

Hi @balazsdukai, I made the following code from your response:

import cjio.cityjson as cj

cms = []

fnames = [
    "F:/CJ/8-504-584.city.json",
    "F:/CJ/8-504-592.city.json",
    "F:/CJ/9-512-584.city.json",
    "F:/CJ/9-512-592.city.json"
]

# Load the first file and check for transform
cm = cj.load(fnames[0], transform=True)
print(f"Loaded a citymodel with {len(cm.cityobjects)} CityObjects")
print(cm.transform)

for p in fnames[1:]:
    _cm = cj.load(p, transform=True)
    print(f"Loaded a citymodel with {len(_cm.cityobjects)} CityObjects")
    print(_cm.transform)
    cms.append(_cm)
    
cm.merge(cms)

(venv) PS F:\GitHub\merge\venv\scripts> & F:/GitHub/merge/venv/Scripts/python.exe f:/GitHub/merge/merge.py
Loaded a citymodel with 1965 CityObjects
{'scale': [0.001, 0.001, 0.001], 'translate': [140599.5625, 453892.6875, 14.80350112915039]}
Loaded a citymodel with 453 CityObjects
{'scale': [0.001, 0.001, 0.001], 'translate': [140569.71875, 455856.0625, 52.490501403808594]}
Loaded a citymodel with 4947 CityObjects
{'scale': [0.001, 0.001, 0.001], 'translate': [142089.640625, 453381.71875, 13.485000610351562]}
Loaded a citymodel with 68 CityObjects
{'scale': [0.001, 0.001, 0.001], 'translate': [142094.984375, 455406.625, 8.404500007629395]}
Traceback (most recent call last):
  File "f:\GitHub\merge\merge.py", line 24, in <module>
    cm.merge(cms)
    ~~~~~~~~^^^^^
  File "F:\GitHub\merge\venv\Lib\site-packages\cjio\cityjson.py", line 1263, in merge
    imp_digits = math.ceil(abs(math.log(self.j["transform"]["scale"][0], 10)))
                                        ~~~~~~^^^^^^^^^^^^^
KeyError: 'transform'

I suspect there is not an issue with the code itself, but rather with a dependency or module, since it has worked fine in the past on another machine. I just cannot figure out what it would be or why.

[balazsdukai]

Note that when you load the base citymodel that is merged-to, in cm = cj.load(fnames[0], transform=True) the transform parameter has to be False. See the first load command in my example.
Setting transform to True will transform the vertices and remove the transformation parameters from the citymodel, that's why your code doesn't find it.

[rob-scholten95]

Note that when you load the base citymodel that is merged-to, in cm = cj.load(fnames[0], transform=True) the transform parameter has to be False.

Unfortunately changing this to False has no effect on the output of the code. It still produces the same error.

EDIT:
After some more testing and digging, i found out that when i create venv, it defaults the interpreter in VSCode to Python 3.12.7, but when i do Python --version in the venv, it outputs Python 3.12.8. If i change the interpreter to 3.12.8 (Microsoft Store), it actually works. However, it only works when i select the Microsoft Store interpreter, not any venv interpreter.