Multiple bugs with pwnlib.encoders

[caprinux]

Was messing around with Pwnlib.encoders and I tried encoding bytes from \x10 through \xff. Found a few bugs along the way

context.arch = 'amd64'

shellcode = b'\x31\xF6\x56\x48\xBB\x2F\x62\x69\x6E\x2F\x2F\x73\x68\x53\x54\x5F\xF7\xEE\xB0\x3B\x0F\x05'
avoid = {b'\x10', b'\x11', b'\x12', b'\x13', b'\x14', b'\x15', b'\x16', b'\x17', b'\x18', b'\x19', b'\x1a', b'\x1b', b'\x1c', b'\x1d', b'\x1e', b'\x1f', b'\x20', b'\x21', b'\x22', b'\x23', b'\x24', b'\x25', b'\x26', b'\x27', b'\x28', b'\x29', b'\x2a', b'\x2b', b'\x2c', b'\x2d', b'\x2e', b'\x2f', b'\x30', b'\x31', b'\x32', b'\x33', b'\x34', b'\x35', b'\x36', b'\x37', b'\x38', b'\x39', b'\x3a', b'\x3b', b'\x3c', b'\x3d', b'\x3e', b'\x3f', b'\x40', b'\x41', b'\x42', b'\x43', b'\x44', b'\x45', b'\x46', b'\x47', b'\x48', b'\x49', b'\x4a', b'\x4b', b'\x4c', b'\x4d', b'\x4e', b'\x4f', b'\x50', b'\x51', b'\x52', b'\x53', b'\x54', b'\x55', b'\x56', b'\x57', b'\x58', b'\x59', b'\x5a', b'\x5b', b'\x5c', b'\x5d', b'\x5e', b'\x5f', b'\x60', b'\x61', b'\x62', b'\x63', b'\x64', b'\x65', b'\x66', b'\x67', b'\x68', b'\x69', b'\x6a', b'\x6b', b'\x6c', b'\x6d', b'\x6e', b'\x6f', b'\x70', b'\x71', b'\x72', b'\x73', b'\x74', b'\x75', b'\x76', b'\x77', b'\x78', b'\x79', b'\x7a', b'\x7b', b'\x7c', b'\x7d', b'\x7e', b'\x7f', b'\x80', b'\x81', b'\x82', b'\x83', b'\x84', b'\x85', b'\x86', b'\x87', b'\x88', b'\x89', b'\x8a', b'\x8b', b'\x8c', b'\x8d', b'\x8e', b'\x8f', b'\x90', b'\x91', b'\x92', b'\x93', b'\x94', b'\x95', b'\x96', b'\x97', b'\x98', b'\x99', b'\x9a', b'\x9b', b'\x9c', b'\x9d', b'\x9e', b'\x9f', b'\xa0', b'\xa1', b'\xa2', b'\xa3', b'\xa4', b'\xa5', b'\xa6', b'\xa7', b'\xa8', b'\xa9', b'\xaa', b'\xab', b'\xac', b'\xad', b'\xae', b'\xaf', b'\xb0', b'\xb1', b'\xb2', b'\xb3', b'\xb4', b'\xb5', b'\xb6', b'\xb7', b'\xb8', b'\xb9', b'\xba', b'\xbb', b'\xbc', b'\xbd', b'\xbe', b'\xbf', b'\xc0', b'\xc1', b'\xc2', b'\xc3', b'\xc4', b'\xc5', b'\xc6', b'\xc7', b'\xc8', b'\xc9', b'\xca', b'\xcb', b'\xcc', b'\xcd', b'\xce', b'\xcf', b'\xd0', b'\xd1', b'\xd2', b'\xd3', b'\xd4', b'\xd5', b'\xd6', b'\xd7', b'\xd8', b'\xd9', b'\xda', b'\xdb', b'\xdc', b'\xdd', b'\xde', b'\xdf', b'\xe0', b'\xe1', b'\xe2', b'\xe3', b'\xe4', b'\xe5', b'\xe6', b'\xe7', b'\xe8', b'\xe9', b'\xea', b'\xeb', b'\xec', b'\xed', b'\xee', b'\xef', b'\xf0', b'\xf1', b'\xf2', b'\xf3', b'\xf4', b'\xf5', b'\xf6', b'\xf7', b'\xf8', b'\xf9', b'\xfa', b'\xfb', b'\xfc', b'\xfd', b'\xfe', b'\xff'}
print(encode(shellcode, avoid))
# OUTPUT: '1\xc3\xb6VH\xc2\xbb/bin//shST_\xc3\xb7\xc3\xae\xc2\xb0;\x0f\x05'

---> 72     if not (force or avoid & set(raw_bytes)):
     73         return raw_bytes

ipdb> avoid & set(raw_bytes)
set()

ipdb> raw_bytes
b'1\xf6VH\xbb/bin//shST_\xf7\xee\xb0;\x0f\x05'
ipdb> set(raw_bytes)
{5, 15, 47, 176, 49, 59, 187, 72, 83, 84, 86, 95, 98, 104, 105, 110, 238, 115, 246, 247}

Iterating over the bytes object yields integers instead of single-byte bytes object.

There's a separate bug if you encode(..., force=True) that complains about calling bytes(avoid)

ipdb> bytes(avoid)
*** TypeError: 'bytes' object cannot be interpreted as an integer

Also this should only be done if context.randomize == True

random.shuffle(encoders)

And this seems to hang forever, even if you do the "right thing"

ipdb> raw_bytes
b'1\xf6VH\xbb/bin//shST_\xf7\xee\xb0;\x0f\x05'
ipdb> bytes(b''.join(avoid))
b'\xdc\x85\xf3\xf7\xcf\xde\x14="+\xd1,\xb9\x9c\xb0\x80\xa9QhX\x9d\x99\xbf\x8b\xbd\xe5\xed7\xe8\x7f:B\x82o\xc8\xc2n\x98\x1cG\xd0\xb6\xf0~\xe3#&u\xfbY]\x89\xc4_\x1dl^\xa8)6E\xe9<R\xb5|\xfc1v}9\x91\x97I\x96\x13!t\xc7\xb4\xd8\xfeZw\x1b\x8d;\xb2F\xeb\xaf\xa3>\xc1\xcbi\x81$\x875T\xa4\xe00\x9bUya\x9e\xee8\x19/b\xa0-\x9fD\xb33\xe4rP\xce\xae\xaaqW[\xfa\xe2\xf1\xfd\xff\xcd\xf6\xb7s?\x8a.\xc0\xca\xd2\xd5\x1ec\xac\x8e\xad\x18A\x11\xc94\\\xdd\xcc@J\xe1\xa6\xdaC\xa7fk\xb1\xd6H\xf5S\x9a\x86\x10\x95\x8c\x83\xb8\xea\xa1Op\xab\xba%\x17Vj \x16M\x93\xe6N\xc5\xf2\xefzK\xd4\xdb\xf4d\x84\x94\x92\x1a\'\x88`\xd9(\x15\xc32me\xf9\xbex\x90L\xe7*\xdf\xa5g\xa2\xbc{\x8f\xc6\x1f\xbb\xf8\xec\xd3\xd7\x12'
ipdb> encoder(raw_bytes, bytes(b''.join(avoid), pcreg)

In summary, there's

• a bunch of bugs that need to be fixed to use bytes objects with encode and friends
• e.g. set(b'asdf') == {97, 115, 100, 102}
• and the docs need to be updated to say bytes instead of str
• and the examples need to be updated to not use a set() apparently
• encoders should not be shuffled unless context.randomize is set

[Arusekk]

FYI it is way better to split such issues, because every encoder we have is (sadly) a separate being.

[heapcrash]

@Arusekk most of these issues are in the core encoders.enode() function, not the individual encoders themselves ☹️

[heapcrash]

See #1923