Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Truncate fields too large for indexes. #80

Open
malcyL opened this issue Jul 15, 2015 · 9 comments
Open

Truncate fields too large for indexes. #80

malcyL opened this issue Jul 15, 2015 · 9 comments

Comments

@malcyL
Copy link
Contributor

malcyL commented Jul 15, 2015

The following error occurred when generating a table row for a field containing a long string:

Exception MongoCursorException
Error
insertDocument :: caused by :: 17280 Btree::insert: key too large to index, failing kent.table_rows.$value_title_1 1085 { : "..." } 

Tripod should check the length of fields being inserted and truncate where necessary.

@rgoj
Copy link

rgoj commented Jul 20, 2015

Today we had another failure on the queue, so increasing the related PagerDuty alert to 2 items on the failed queue.

@scaleupcto
Copy link
Contributor

@malcyL I think my preferred approach to fixing this is when the exception happens, we look at which fields are indexed on table rows (this is in the config) and truncate all those strings to less than < 1024 bytes

@scaleupcto
Copy link
Contributor

@Roj this is a public repo, your comment feels like it should live on our private repo as it is specific to our use of tripod and not relevant to anyone outside Talis ;-)

@rsinger
Copy link
Member

rsinger commented Jul 21, 2015

Do we know the exception that gets thrown in this instance? We need to be sure we're only catching the most specific exception we can.

@scaleupcto
Copy link
Contributor

I think it's a MongoCursorException, message shown above

@rsinger
Copy link
Member

rsinger commented Jul 21, 2015

Oh, duh - right. Sorry. Bah, that's not terribly granular.

@scaleupcto
Copy link
Contributor

Nope, but I guess a strpos($msg,"Btree::insert: key too large to index")!==FALSE would do it.

@rsinger
Copy link
Member

rsinger commented Jul 21, 2015

Yep, seems like a reasonable approach.

@malcyL
Copy link
Contributor Author

malcyL commented Jul 22, 2015

Looking further into this, this is only failing in Mongo 2.6 and above.

See https://jira.mongodb.org/browse/SERVER-12982 - stricter enforcement of the handling of the index key limit.

However, see https://jira.mongodb.org/browse/SERVER-12834 - which adds a flag which if set allows mongo to ignore this error. This was added for people who already had indexes containing keys which exceed the limit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants