Malformed Packet error querying Sphinx 2.1.1 server...

[jzawodn]

I'm using this simple program to test out the sphinxql driver (not sure if it's ready for prime time given the lack of docs, but figured I'd give it a spin since the mysql driver doesn't talk to sphinxql):

package main

import "fmt"
import "database/sql"
import _ "github.com/go-sql-driver/sphinxql"

func main() {

    db, err := sql.Open("sphinxql", "user:pass@tcp(dev7h:9306)/")
    fmt.Println("opened!")

    if err != nil {
        panic(err.Error())
    }

    defer db.Close()

    rows, err := db.Query("SELECT id FROM index_1 LIMIT 10")

    if err != nil {
        panic(err.Error())
    }

    for rows.Next() {
        var id string
            if err := rows.Scan(&id); err != nil {
                fmt.Println(err.Error())
                return
            }
        fmt.Println(id)
    }

}

Upon running, I get this error"

$ ./sphinxql-test
opened!
panic: Malformed Packet

goroutine 1 [running]:
main.main()
    /Users/jzawodn/code/go/mysql/sphinxql-test.go:21 +0x224

goroutine 2 [syscall]:

Line 21 is the db.Query() call.

Any thoughts?

[julienschmidt]

Sorry, I have to disappoint you. The driver is not ready yet.
I'll merge it with upstream (the mysql driver) soon. I think the bug is fixed in upstream but I must admit that I currently have no test environment running.

[julienschmidt]

I haven't tested it, but I was told the mysql driver works with Sphinx if you are using Go 1.1 and the query has no parameters. Sphinx doesn't support the binary protocol which is used in the mysql driver for queries with args. You can easily build such a query string without args with fmt.Printf. But take care with strings which should be escaped.

[jzawodn]

Interesting. I initially tried with the MySQL driver, but ran into an issue with the ping call (it appeared to just hang). I'll do a bit more testing with it then and see if I can make it work. My test program was similarly simple (no args).

Thanks for the quick feedback.

If I find an issue with it, should i open an issue on the MySQL driver then? Or do you have plans to continue on with this SphinxQL driver?

[julienschmidt]

Please just comment on this issue here. There are quite a few people watching the mysql repo which I don't want to disturb with not really related mail notifications.
I think Sphinx won't support most of the DSN parameters of the mysql driver but except for the also missing binary protocol support it should be mostly the same.
The plan is to build a sphinxql driver from a stable code base of the mysql driver. Version 1.1 of the mysql driver will be released soon and soon after that release I'll branch of the sphinxql driver then.

[jzawodn]

Ok, I've tried a bunch of different things using the MySQL driver and the symptom is the same: I'm able to get a connection (verified with netstat) but calling Query() fails to send the query and the program hangs there.

Here's an example:

package main

import "fmt"
import "database/sql"
import _ "github.com/go-sql-driver/mysql"

func main() {

    db, err := sql.Open("mysql", "user:pass@tcp(dev7h:9306)/")
    fmt.Println("opened!")

    if err != nil {
        panic(err.Error())
    }

    defer db.Close()

    fmt.Println("starting query")
    //rows, err := db.Query("SELECT * FROM index_1 LIMIT 10")
    //rows, err := db.Query("SHOW VARIABLES")
    //queryString := "SHOW VARIABLES"
    //queryString := "SHIT VARIABLES"
    db.Query("")
    fmt.Println("query done")
}

It seems not to matter what string I use in the db.Query() call, I'll never see the "query done" message. You can see the other commented out attempts I made.

[julienschmidt]

sql.Open doesn't open a connection, see: https://github.com/go-sql-driver/mysql/wiki/Examples#a-word-on-sqlopen

Does it fail already if you just do db.Ping()? This opens a connection.

[jzawodn]

The db.Ping() call never returns. Here's the code I'm using now:

package main

import "fmt"
import "database/sql"
import _ "github.com/go-sql-driver/mysql"

func main() {

    db, err := sql.Open("mysql", "no:no@tcp(dev4h:9306)/")
    fmt.Println("opened!")

    if err != nil {
        panic(err.Error())
    }

    defer db.Close()

    fmt.Println("starting ping")
    db.Ping()
    fmt.Println("ping gone")

    fmt.Println("starting query")
    rows, err := db.Query("SHOW VARIABLES")

    if err != nil {
        panic(err.Error())
    }
    fmt.Println("query done")

    for rows.Next() {
        var id string
            if err := rows.Scan(&id); err != nil {
                fmt.Println(err.Error())
                return
            }
        fmt.Println(id)
    }
}

The output stops after "starting ping" and just hangs. If I run a "netstat -an | grep 9306" on the remote host, I see the established connection.

Similar things happen when I try to execute a query instead of pinging, it'll hang after "starting query" and the connection is open. But the query is never logged on the remote end, which makes me thing it isn't transmitted or isn't transmitted in a way that the remote server sees as being a complete query.

[jzawodn]

If it's helpful, I ran that and let it hang at the Ping() call. While it was hanging, I stopped sphinx on the remote host and this was the output:

$ ./test2
opened!
starting ping
[MySQL] 2013/07/05 10:53:18 packets.go:31: EOF
ping gone
starting query
panic: dial tcp 172.17.14.1:9306: connection refused

goroutine 1 [running]:
main.main()
    /Users/jzawodn/code/go/mysql/test2.go:26 +0x418

goroutine 2 [syscall]:

I'd look at packets.go but I don't know the mysql protocol well enough to attempt diagnosing it.

[martingallagher]

Howdy, I've come up with a "fix" - which is to remove the check on packets.go line 138.

Returning prematurely before "return errMalformPkt" allows me to interact with the Sphinx server without issue (Sphinx: Server version: 2.2.1-id64-dev (r4003))

It seems:

    // The documentation is ambiguous about the length.
    // The official Python library uses the fixed length 12
    // which is not documented but seems to work.

Is naive in this instance. Hopefully this hack will let you connect successfully with Sphinx!

[martingallagher]

Turns out the hack allows you to connect and perform DML queries (UPDATE/INSERT/DELETE) but not SELECTs. Malformed packet errors emitted. Any hints on what could be causing this?

[julienschmidt]

We fixed related bugs in the MySQL code a while back. I think a merge with the upstream code might help. I'll have time for that next week.

[martingallagher]

Cheers, in the meantime, I removed the check in readResultSetHeaderPacket() and SELECT statements work as expected:

// column count
num, _, _ := readLengthEncodedInteger(data)

//if n-len(data) == 0 {
return int(num), nil
//}

[martingallagher]

Any update? I see the mysql driver has received some attention lately, cheers!

[jessp01]

I know this is a very old issue but for the benefit of whoever hits it: I forked the original sphinxql [now deprecated] repo and patched it:
https://github.com/jessp01/sphinxql

This was tested against Sphinx 2.2.1. Needless to say, if you've got Sphinx >= 2.2.3, you should use:
https://github.com/go-sql-driver/mysql.