OperationalError: Could not decode to UTF-8 column 'value' with text 'Johan Otterström'

[Antti Jääskö]

How to Reproduce

While doing a GET operation on /admin/accounts/users, Trac issued an internal error.

Trac was upgraded from 0.12 to 1.2.2. Then I enabled TracAccountManager plugin to get Users (under Accounts) visible. But when clicking "Users" then that OperationalError was printed out.

Request parameters:

{'cat_id': u'accounts', 'panel_id': u'users', 'path_info': None}

User agent: Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; .NET4.0C; .NET4.0E; rv:11.0) like Gecko

System Information

Trac	1.2.2
Babel	2.5.1
CustomFieldAdmin	0.2.5
Genshi	0.7 (with speedups)
pysqlite	2.6.0
Python	2.7.5 (default, May 3 2017, 07:55:04) [GCC 4.8.5 20150623 (Red Hat 4.8.5-14)]
pytz	2016.10
setuptools	0.9.8
SQLite	3.7.17
jQuery	1.11.3
jQuery UI	1.11.4
jQuery Timepicker	1.5.5

Enabled Plugins

AutocompleteUsers	0.4.1
ticket-clone	N/A
TicketImport	0.8
TracAccountManager	0.5dev
TracCustomFieldAdmin	0.2.5
TracDateField	1.0.1
TracDynamicFields	2.2.0dev
TypedTicketWorkflow-plugin	0.1dev-r4791

Interface Customization

shared-htdocs
shared-templates	site.html
site-htdocs
site-templates

Python Traceback

Traceback (most recent call last):
  File "build/bdist.linux-x86_64/egg/trac/web/main.py", line 623, in _dispatch_request
    dispatcher.dispatch(req)
  File "build/bdist.linux-x86_64/egg/trac/web/main.py", line 239, in dispatch
    resp = chosen_handler.process_request(req)
  File "build/bdist.linux-x86_64/egg/trac/admin/web_ui.py", line 109, in process_request
    provider.render_admin_panel(req, cat_id, panel_id, path_info)
  File "build/bdist.linux-x86_64/egg/acct_mgr/admin.py", line 218, in render_admin_panel
    return self._do_users(req)
  File "build/bdist.linux-x86_64/egg/acct_mgr/admin.py", line 692, in _do_users
    filters)))
  File "build/bdist.linux-x86_64/egg/acct_mgr/admin.py", line 65, in fetch_user_data
    authenticated=None).iteritems():
  File "build/bdist.linux-x86_64/egg/acct_mgr/model.py", line 441, in get_user_attribute
    for row in env.db_query(sql, sql_args):
  File "build/bdist.linux-x86_64/egg/trac/db/api.py", line 122, in execute
    return db.execute(query, params)
  File "build/bdist.linux-x86_64/egg/trac/db/util.py", line 128, in execute
    cursor.execute(query, params if params is not None else [])
  File "build/bdist.linux-x86_64/egg/trac/db/util.py", line 73, in execute
    return self.cursor.execute(sql)
  File "build/bdist.linux-x86_64/egg/trac/db/sqlite_backend.py", line 89, in execute
    self.rows = PyFormatCursor.fetchall(self)
OperationalError: Could not decode to UTF-8 column 'value' with text 'Johan Otterström'

[Jun Omae]

AccountManagerPlugin 0.4.1 doesn't work with Trac 1.2. See AccountManagerPlugin#DownloadsandSource.

Please confirm it with AccountManagerPlugin trunk.

[Jun Omae]

Replying to Jun Omae:

AccountManagerPlugin 0.4.1 doesn't work with Trac 1.2. See AccountManagerPlugin#DownloadsandSource.

Sorry about my misreading. Please ignore it.

[Jun Omae]

It seems the exception is raised when stored bytes string is unable to fail to decode to unicode. So, I guess text in the value column is not a valid utf-8 string.

See ​https://github.com/ghaering/pysqlite/blob/2.6.0/src/cursor.c#L389.

TEXT column in SQLite can have an invalid utf-8 string (x'F020F0' is invalid utf-8 string).

sqlite> INSERT INTO session_attribute (sid, authenticated, name, value)
   ...> VALUES ('blah', 1, 'name', x'F020F0');
sqlite> SELECT sid, authenticated, name, value, HEX(value)
   ...> FROM session_attribute
   ...> WHERE sid='blah' AND authenticated=1 AND name='name';
blah|1|name|� �|F020F0

Could you please post results of the following query in your trac.db?

SELECT sid, authenticated, name, value, HEX(value)
FROM session_attribute
WHERE value GLOB '*Johan*Otter*';

[anonymous]

Replying to Jun Omae:

Hi,

I forwarded your query to our Trac server admin and I will get back to you as soon as possible.

Regards, Antti

[Jun Omae]

No feedback, closing.