Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

IOU VM "IOU1" process has stopped with return code: -6 (IOU L2 v17.12.1) #868

Closed
Zero3Dash opened this issue Mar 29, 2024 · 21 comments
Closed

IOU VM "IOU1" process has stopped with return code: -6 (IOU L2 v17.12.1) #868

Zero3Dash opened this issue Mar 29, 2024 · 21 comments

Comments

@Zero3Dash
Copy link
Contributor

https://github.com/GNS3/gns3-registry/blob/master/appliances/cisco-iou-l2.gns3a

The Cisco IOU L2 image (IOU L2 v17.12.1) for x86_64_crb_linux_l2-adventerprisek9-ms presents IOU VM "IOU1" process has stopped with return code: -6 when attempting to run with the default options.
@Zero3Dash
Copy link
Contributor Author

Running on VMware ESXi Server version 2.2.46

@grossmj
Copy link
Member

grossmj commented Mar 29, 2024

The Cisco IOU L2 image (IOU L2 v17.12.1) for x86_64_crb_linux_l2-adventerprisek9-ms presents IOU VM "IOU1" process has stopped with return code: -6 when attempting to run with the default options.

Maybe more RAM is needed? @Desposyni do you know?

@Zero3Dash
Copy link
Contributor Author

I've tried 256MB, 512MB , 768MB and 1028MB. Same result.

The layer 3 (router) image of the same version number works fine with 256 MB.

@Desposyni
Copy link
Contributor

Desposyni commented Mar 29, 2024 •
edited
Loading

It's not the RAM, the L2-IOU only consumes around 150MB when idle, and as long as the 'use default memory' is checked the IOU file will choose to use 768MB as its max. It's something with the GNS3 VM. The L2-IOU works with Mint and Pop!_OS. Not sure if it's the kernel or something else within the distro.
When running the files directly with -d and -N flags, the L3-IOU and L2-IOU throw the same warnings, but L2 aborts. The only difference is the L3-IOU detects a platform of IOL, while the L2-IOU ends up with a platform of (null).
The first 7 bytes of the file show it's a 64 bit ELF version 1 executable and the file name doesn't start or end with the usual IOU file name, so it also has issues being detected as a legitimate IOU file. I tried editing images.py on my server to make it detect, but no luck. Any ideas as to what dependency the VM is missing that other OSes have and that the L2 needs, but the L3 doesn't?

@Zero3Dash
Copy link
Contributor Author

Zero3Dash commented Mar 30, 2024 •
edited
Loading

I've built a new GNS3 VM using Ubuntu 22.04.04 (fully updated) and GNS3 Server 2.2.46 and the image boots fine.

sysadmin@gns3vm:~$ gns3server
No configuration file could be found or read
2024-03-30 10:49:53 INFO run.py:218 GNS3 server version 2.2.46
2024-03-30 10:49:53 INFO run.py:220 Copyright (c) 2007-2024 GNS3 Technologies Inc.
2024-03-30 10:49:53 INFO run.py:242 Running with Python 3.10.12 and has PID 1251
2024-03-30 10:49:53 INFO run.py:248 Using system certificate store for SSL connections
2024-03-30 10:49:53 INFO run.py:78 Current locale is en_US.UTF-8
2024-03-30 10:49:53 INFO web_server.py:338 Starting server on 0.0.0.0:3080
2024-03-30 10:49:53 INFO init.py:70 Load controller configuration file /home/sysadmin/.config/GNS3/2.2/gns3_controller.conf
2024-03-30 10:49:53 INFO init.py:74 Controller is starting
2024-03-30 10:49:53 INFO init.py:314 Installing base configs in '/home/sysadmin/GNS3/configs'
2024-03-30 10:49:53 INFO init.py:327 Installing built-in disks in '/home/sysadmin/GNS3/images/QEMU'
2024-03-30 10:49:53 INFO init.py:252 Comparing controller version 2.2.46 with config version 2.2.46
2024-03-30 10:49:53 INFO compute.py:64 Create compute local
2024-03-30 10:49:53 INFO compute.py:364 Connecting to compute 'local'
2024-03-30 10:49:53 INFO web_log.py:211 127.0.0.1 [30/Mar/2024:10:49:53 +0000] "GET /v2/compute/capabilities HTTP/1.1" 200 553 "-" "Python/3.10 aiohttp/3.9.3"
2024-03-30 10:49:53 INFO web_server.py:247 Computing image checksums...
2024-03-30 10:49:53 INFO notification_handler.py:50 New client has connected to compute WebSocket
2024-03-30 10:49:53 INFO web_server.py:249 Finished computing image checksums
2024-03-30 10:49:53 INFO compute.py:431 Connected to compute 'local' WebSocket 'http://127.0.0.1:3080/v2/compute/notifications/ws'
2024-03-30 10:49:57 INFO web_log.py:211 192.168.1.50 [30/Mar/2024:10:49:57 +0000] "GET /v2/version HTTP/1.1" 200 229 "-" "GNS3 QT Client v2.2.46"
2024-03-30 10:49:57 INFO notification_handler.py:73 New client has connected to controller WebSocket
2024-03-30 10:49:57 INFO web_log.py:211 192.168.1.50 [30/Mar/2024:10:49:57 +0000] "GET /v2/templates HTTP/1.1" 200 3976 "-" "GNS3 QT Client v2.2.46"
2024-03-30 10:49:57 INFO web_log.py:211 192.168.1.50 [30/Mar/2024:10:49:57 +0000] "GET /v2/computes HTTP/1.1" 200 1022 "-" "GNS3 QT Client v2.2.46"
2024-03-30 10:49:57 INFO web_log.py:211 192.168.1.50 [30/Mar/2024:10:49:57 +0000] "GET /v2/projects HTTP/1.1" 200 832 "-" "GNS3 QT Client v2.2.46"
2024-03-30 10:49:57 INFO web_log.py:211 192.168.1.50 [30/Mar/2024:10:49:57 +0000] "GET /v2/version HTTP/1.1" 200 229 "-" "GNS3 QT Client v2.2.46"
2024-03-30 10:49:57 INFO web_log.py:211 192.168.1.50 [30/Mar/2024:10:49:57 +0000] "GET /v2/appliances?symbol_theme=Affinity-circle-blue HTTP/1.1" 200 999047 "-" "GNS3 QT Client v2.2.46"
2024-03-30 10:49:57 INFO project.py:78 Project 9ac13950-01c8-492b-83ac-fb65fb205c75 with path '/home/sysadmin/GNS3/projects/9ac13950-01c8-492b-83ac-fb65fb205c75' created
2024-03-30 10:49:57 INFO web_log.py:211 127.0.0.1 [30/Mar/2024:10:49:57 +0000] "POST /v2/compute/projects HTTP/1.1" 201 300 "-" "Python/3.10 aiohttp/3.9.3"
2024-03-30 10:49:57 INFO iou_vm.py:73 IOU "IOU1" [c0824fb6-3d31-44b8-bea5-8719fe874952]: assigned with application ID 1
2024-03-30 10:49:57 INFO iou_vm.py:875 IOU "IOU1" [c0824fb6-3d31-44b8-bea5-8719fe874952]: number of Ethernet adapters changed to 2
2024-03-30 10:49:57 INFO iou_vm.py:903 IOU "IOU1" [c0824fb6-3d31-44b8-bea5-8719fe874952]: number of Serial adapters changed to 2
IOS On Unix - Cisco Systems confidential, internal use only
2024-03-30 10:49:57 INFO iou_vm.py:288 IOU "IOU1" [c0824fb6-3d31-44b8-bea5-8719fe874952]: RAM updated from 256MB to 768MB
2024-03-30 10:49:57 INFO iou_vm.py:875 IOU "IOU1" [c0824fb6-3d31-44b8-bea5-8719fe874952]: number of Ethernet adapters changed to 4
2024-03-30 10:49:57 INFO iou_vm.py:145 IOU "IOU1" [c0824fb6-3d31-44b8-bea5-8719fe874952]: IOU image updated to "/home/sysadmin/GNS3/images/IOU/x86_64_crb_linux_l2-adventerprisek9-ms"
2024-03-30 10:49:57 INFO iou_vm.py:903 IOU "IOU1" [c0824fb6-3d31-44b8-bea5-8719fe874952]: number of Serial adapters changed to 0
2024-03-30 10:49:57 INFO web_log.py:211 127.0.0.1 [30/Mar/2024:10:49:57 +0000] "POST /v2/compute/projects/9ac13950-01c8-492b-83ac-fb65fb205c75/iou/nodes HTTP/1.1" 201 899 "-" "Python/3.10 aiohttp/3.9.3"
2024-03-30 10:49:57 INFO web_log.py:211 192.168.1.50 [30/Mar/2024:10:49:57 +0000] "POST /v2/projects/9ac13950-01c8-492b-83ac-fb65fb205c75/open HTTP/1.1" 201 767 "-" "GNS3 QT Client v2.2.46"
2024-03-30 10:49:57 INFO project_handler.py:272 New client has connected to the notification stream for project ID '9ac13950-01c8-492b-83ac-fb65fb205c75' (WebSocket method)
2024-03-30 10:49:57 INFO web_log.py:211 192.168.1.50 [30/Mar/2024:10:49:57 +0000] "GET /v2/projects/9ac13950-01c8-492b-83ac-fb65fb205c75/nodes HTTP/1.1" 200 6922 "-" "GNS3 QT Client v2.2.46"
2024-03-30 10:49:57 INFO web_log.py:211 192.168.1.50 [30/Mar/2024:10:49:57 +0000] "GET /v2/projects/9ac13950-01c8-492b-83ac-fb65fb205c75/links HTTP/1.1" 200 203 "-" "GNS3 QT Client v2.2.46"
2024-03-30 10:49:57 INFO web_log.py:211 192.168.1.50 [30/Mar/2024:10:49:57 +0000] "GET /v2/projects/9ac13950-01c8-492b-83ac-fb65fb205c75/drawings HTTP/1.1" 200 206 "-" "GNS3 QT Client v2.2.46"
2024-03-30 10:50:00 INFO web_log.py:211 192.168.1.50 [30/Mar/2024:10:50:00 +0000] "GET /v2/gns3vm HTTP/1.1" 200 378 "-" "GNS3 QT Client v2.2.46"
2024-03-30 10:50:00 INFO web_log.py:211 192.168.1.50 [30/Mar/2024:10:50:00 +0000] "GET /v2/iou_license HTTP/1.1" 200 277 "-" "GNS3 QT Client v2.2.46"
2024-03-30 10:50:00 INFO init.py:70 Load controller configuration file /home/sysadmin/.config/GNS3/2.2/gns3_controller.conf
2024-03-30 10:50:00 INFO web_log.py:211 192.168.1.50 [30/Mar/2024:10:50:00 +0000] "GET /v2/gns3vm/engines HTTP/1.1" 200 1317 "-" "GNS3 QT Client v2.2.46"
2024-03-30 10:50:00 INFO web_log.py:211 192.168.1.50 [30/Mar/2024:10:50:00 +0000] "GET /v2/gns3vm/engines/vmware/vms HTTP/1.1" 200 203 "-" "GNS3 QT Client v2.2.46"
2024-03-30 10:50:04 INFO init.py:70 Load controller configuration file /home/sysadmin/.config/GNS3/2.2/gns3_controller.conf
2024-03-30 10:50:04 INFO web_log.py:211 192.168.1.50 [30/Mar/2024:10:50:04 +0000] "PUT /v2/iou_license HTTP/1.1" 201 282 "-" "GNS3 QT Client v2.2.46"
2024-03-30 10:50:07 INFO iou_vm.py:414 Checking IOU license in '/tmp/tmpv0dr5s34/iourc'
2024-03-30 10:50:07 INFO base_node.py:701 Starting new uBridge hypervisor 0.0.0.0:34487
2024-03-30 10:50:07 INFO hypervisor.py:169 starting ubridge: ['/usr/bin/ubridge', '-H', '0.0.0.0:34487']
2024-03-30 10:50:07 INFO hypervisor.py:171 logging to /home/sysadmin/GNS3/projects/9ac13950-01c8-492b-83ac-fb65fb205c75/project-files/iou/c0824fb6-3d31-44b8-bea5-8719fe874952/ubridge.log
2024-03-30 10:50:07 INFO hypervisor.py:179 ubridge started PID=1272
2024-03-30 10:50:07 INFO base_node.py:704 Hypervisor 0.0.0.0:34487 has successfully started
2024-03-30 10:50:07 INFO ubridge_hypervisor.py:83 Connected to uBridge hypervisor on 127.0.0.1:34487 after 0.1007 seconds
2024-03-30 10:50:07 INFO iou_vm.py:780 IOU IOU1 [id=c0824fb6-3d31-44b8-bea5-8719fe874952]: NETMAP file created
IOS On Unix - Cisco Systems confidential, internal use only
2024-03-30 10:50:07 INFO iou_vm.py:556 Starting IOU: ['/home/sysadmin/GNS3/images/IOU/x86_64_crb_linux_l2-adventerprisek9-ms', '-e', '4', '-s', '0', '1']
2024-03-30 10:50:07 INFO iou_vm.py:565 IOU instance c0824fb6-3d31-44b8-bea5-8719fe874952 started PID=1277
2024-03-30 10:50:07 INFO web_log.py:211 127.0.0.1 [30/Mar/2024:10:50:07 +0000] "POST /v2/compute/projects/9ac13950-01c8-492b-83ac-fb65fb205c75/iou/nodes/c0824fb6-3d31-44b8-bea5-8719fe874952/start HTTP/1.1" 200 991 "-" "Python/3.10 aiohttp/3.9.3"
2024-03-30 10:50:07 INFO web_log.py:211 192.168.1.50 [30/Mar/2024:10:50:07 +0000] "POST /v2/projects/9ac13950-01c8-492b-83ac-fb65fb205c75/nodes/c0824fb6-3d31-44b8-bea5-8719fe874952/start HTTP/1.1" 200 6187 "-" "GNS3 QT Client v2.2.46"
2024-03-30 10:50:11 INFO iou_vm.py:1295 saving startup-config to /home/sysadmin/GNS3/projects/9ac13950-01c8-492b-83ac-fb65fb205c75/project-files/iou/c0824fb6-3d31-44b8-bea5-8719fe874952/startup-config.cfg
2024-03-30 10:50:11 INFO web_log.py:211 192.168.1.50 [30/Mar/2024:10:50:11 +0000] "GET /v2/computes HTTP/1.1" 200 1022 "-" "GNS3 QT Client v2.2.46"
2024-03-30 10:50:15 INFO iou_vm.py:1295 saving startup-config to /home/sysadmin/GNS3/projects/9ac13950-01c8-492b-83ac-fb65fb205c75/project-files/iou/c0824fb6-3d31-44b8-bea5-8719fe874952/startup-config.cfg
2024-03-30 10:50:15 INFO web_log.py:211 192.168.1.50 [30/Mar/2024:10:50:15 +0000] "GET /v2/computes HTTP/1.1" 200 1022 "-" "GNS3 QT Client v2.2.46"

@grossmj
Copy link
Member

grossmj commented Apr 2, 2024

Not sure what happened, return code -6 means that the IOU process failed with SIGABRT. I don't think we can do anything about it.

@Zero3Dash should we close this issue?

@Zero3Dash
Copy link
Contributor Author

@grossmj
I've build 2 VM's on Ubuntu 20.04.04 and 22.04.04 over the weekend.

It runs fine on 22.04.04, but fails with the -6 error code on 20.04.04. Which would indicate its something within the Linux Distro.

If it's not something that will be fix in the here and now, but later when the distro gets upgraded. Then, it might be worth including in the appliance file that the version IOU L2 v17.12.1 will only successfully run on GNS3 running 22.04.04 and possible higher..

I'm happy for you to close it, btw.

@TheLordOfTheKings
Copy link

TheLordOfTheKings commented Apr 5, 2024 •
edited
Loading

Hey guys, @Zero3Dash @grossmj
Please don't close the issue yet,
I have the exact problem with the latest L2 image (17.12.1). Can i upgrade my current GNS3 2.2.46 server with ubuntu 20.04 to 22.04 ? how?
Thanks.

@Zero3Dash
Copy link
Contributor Author

@Arash88
If you are using the pre-compiled VM, you technically do not.

What I did was to create a new VM using Ubuntu Server 22.04 ISO, then manually installed GNS3 2.2.46, using the following guide.
https://docs.gns3.com/docs/getting-started/installation/linux/

@Zero3Dash
Copy link
Contributor Author

closed

@ghost
Copy link

ghost commented Apr 5, 2024

I'm using a GNS3 server running directly on Debian 12 (Bookworm) and I had the same issue. I could resolve it by commenting/deleting the xml.cisco.com entry in my /etc/hosts.

Then I tested the GNS3VM (based on Ubuntu 20.04/focal). Also here the issue was resolved by removing the xml.cisco.com entry in /etc/hosts.

So it is not necessary to switch to a new OS.

@grossmj
Copy link
Member

grossmj commented Apr 5, 2024

@Arash88 please try the suggestion above and let me know if it works for you.

@Zero3Dash Zero3Dash reopened this Apr 5, 2024
@Zero3Dash
Copy link
Contributor Author

@grossmj
It works for me, not tested the impact on other images.

On and unrelated note, I've also notice in the 17.12.1 base startup config file, "no ip domain-lookup" should be "no ip domain lookup"

@ghost
Copy link

ghost commented Apr 5, 2024

It works for me, not tested the impact on other images.

xml.cisco.com is currently not used by cisco, so it's unlikely that it has an impact. And I have noticed no impact on other images.

$ host xml.cisco.com 8.8.8.8
Using domain server:
Name: 8.8.8.8
Address: 8.8.8.8#53
Aliases: 

Host xml.cisco.com not found: 3(NXDOMAIN)

@grossmj
Copy link
Member

grossmj commented Apr 6, 2024

I can read the content inside but can't make a change. Nano opens empty.

That's strange, I don't have this issue.

Screenshot from 2024-04-06 12-08-30

Maybe try with vi?

@TheLordOfTheKings
Copy link

TheLordOfTheKings commented Apr 6, 2024 •
edited
Loading

Thanks @grossmj, chmod and chown helped me edit it.
Everything works fine now! I can confirm this method works. Also we still can block xml.cisco.com without messing things up by adding this rule to iptables:

iptables -I OUTPUT -p udp --dport 53 -m string --hex-string "|03|xml|05|cisco|03|com" --algo bm -j DROP

@grossmj
Copy link
Member

grossmj commented Apr 6, 2024

Also we still can block xml.cisco.com without messing things up by adding this rule to iptables.

I have created this PR GNS3/gns3-server#2366

@ghost
Copy link

ghost commented Apr 14, 2024

@grossmj What about removing xml.cisco.com from /etc/hosts in the gns3-vm? Currently it is set in scripts/provision.sh in every branch of the gns3-vm.

scripts/provision.sh

# Block IOU phone home call
echo "127.0.0.254 xml.cisco.com" | sudo tee --append /etc/hosts

@grossmj
Copy link
Member

grossmj commented Apr 15, 2024

@grossmj What about removing xml.cisco.com from /etc/hosts in the gns3-vm? Currently it is set in scripts/provision.sh in every branch of the gns3-vm.

I am considering it. I haven't removed it yet because the problem only affects Ubuntu Focal, so only older GNS3 VMs. For future releases, should we block xml.cisco.com using iptables instead or totally remove any blocking?

@ghost
Copy link

ghost commented Apr 15, 2024

I haven't removed it yet because the problem only affects Ubuntu Focal, so only older GNS3 VMs.

Just tested the GNSVM image GNS3.VM.KVM.2.2.46.zip, it uses focal. I don't know, what OS the other GNS3 VMs are using, but for me it looks, that focal is still actively distributed as the GNS3 VM base OS.

For future releases, should we block xml.cisco.com using iptables instead or totally remove any blocking?

I'm not sure. The iputils command blocks the DNS request to xml.cisco.com, so that a DNS lookup to that address will time out. That may slow down the IOU startup, but I haven't noticed a delay. If jammy doesn't show the issue, I won't add the iputils command. The xml.cisco.com entry in /etc/hosts should be sufficient.

@grossmj
Copy link
Member

grossmj commented Apr 15, 2024

Just tested the GNSVM image GNS3.VM.KVM.2.2.46.zip, it uses focal. I don't know, what OS the other GNS3 VMs are using, but for me it looks, that focal is still actively distributed as the GNS3 VM base OS.

That's right, I forgot 2.2.x still uses Ubuntu Focal and 3.x uses Jammy.

I think I will just remove xml.cisco.com from the hosts file and not mess with iptables.

grossmj added a commit to GNS3/gns3-vm that referenced this issue Apr 20, 2024
@grossmj
Do not block xml.cisco.com on Ubuntu Focal. Fixes GNS3/gns3-registry#868
213764f
@grossmj grossmj closed this as completed Apr 20, 2024
@grossmj grossmj mentioned this issue Dec 28, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@grossmj @Desposyni @TheLordOfTheKings @Zero3Dash