Modify

Opened 5 years ago

Closed 4 months ago

#10135 closed defect (fixed)

Umlauts doesn't work

Reported by: falkb Owned by: Joachim Hoessler
Priority: normal Component: EstimationToolsPlugin
Severity: normal Keywords:
Cc: Ryan J Ollos, Odd Simon Simonsen Trac Release: 0.12

Description

In function def execute_query(env, req, query_args): of utils.py, I tried to add

  • estimationtoolsplugin/trunk/estimationtools/utils.py

     
    134137                        .replace('+', ' ')\
    135138                        .replace('%23', '#')\
    136139                        .replace('%28', '(')\
    137                         .replace('%29', ')')
     140                        .replace('%29', ')')\
     141                        .replace('%C3%84', 'Ä')\
     142                        .replace('%C3%96', 'Ö')\
     143                        .replace('%C3%9C', 'Ü')\
     144                        .replace('%C3%A4', 'ä')\
     145                        .replace('%C3%B6', 'ö')\
     146                        .replace('%C3%BC', 'ü')\
     147                        .replace('%C3%9F', 'ß')
    138148    env.log.debug("query_string: %s" % query_string)
    139149    query = Query.from_string(env, query_string)

but just get this error:

Error: Macro WorkloadChart(milestone=testgehäuse) failed

You must not use 8-bit bytestrings unless you use a text_factory that can interpret 8-bit bytestrings (like text_factory = str). It is highly recommended that you instead just switch your application to Unicode strings.

How must umlauts be fixed?

Attachments (0)

Change History (11)

comment:1 Changed 5 months ago by anonymous

Cc: rjollos,osimonsrjollos, osimons

It's still not in the trunk, why?

comment:2 Changed 5 months ago by Jun Omae

That patch cannot fix this issue. The same issue would be raised if other unicode characters are used.

Could you please try the following patch?

  • estimationtoolsplugin/trunk/estimationtools/utils.py

    diff --git a/estimationtoolsplugin/trunk/estimationtools/utils.py b/estimationtoolsplugin/trunk/estimationtools/utils.py
    index 76e0d8bf1..b167c6be6 100644
    a b from time import strptime 
    1414from trac.config import Option, ListOption, BoolOption
    1515from trac.core import TracError, Component, implements
    1616from trac.ticket.query import Query
    17 from trac.util.datefmt import from_utimestamp, utc
    18 from trac.util.text import unicode_urlencode
     17from trac.util.datefmt import from_utimestamp
    1918from trac.web.api import IRequestHandler, RequestDone
    2019from trac.wiki.api import parse_args
    2120
    def parse_options(env, content, options): 
    147146def execute_query(env, req, query_args):
    148147    # set maximum number of returned tickets to 0 to get all tickets at once
    149148    query_args['max'] = 0
    150     # urlencode the args, converting back a few vital exceptions:
    151     # see the authorized fields in the query language in
    152     # http://trac.edgewall.org/wiki/TracQuery#QueryLanguage
    153     query_string = unicode_urlencode(query_args).replace('%21=', '!=') \
    154         .replace('%21%7E=', '!~=') \
    155         .replace('%7E=', '~=') \
    156         .replace('%5E=', '^=') \
    157         .replace('%24=', '$=') \
    158         .replace('%21%5E=', '!^=') \
    159         .replace('%21%24=', '!$=') \
    160         .replace('%7C', '|') \
    161         .replace('+', ' ') \
    162         .replace('%23', '#') \
    163         .replace('%28', '(') \
    164         .replace('%29', ')')
     149    query_string = '&'.join('%s=%s' % item for item in query_args.iteritems())
    165150    env.log.debug("query_string: %s", query_string)
    166151    query = Query.from_string(env, query_string)
    167152

comment:3 Changed 5 months ago by falkb

Jun, your patch works well. Thanks!

comment:4 Changed 4 months ago by anonymous

Could you commit the patch, please! And close this ticket then.

comment:5 Changed 4 months ago by falkb

I tried to commit the patch above but have no permissions. So I'm giving up with letting you know this could have been fixed. Neither can I set Jun Omae on CC of this ticket.

comment:6 Changed 4 months ago by Ryan J Ollos

Resolution: fixed
Status: newclosed

In 16736:

EstimationTools 0.5.0dev: Fix incorrect encoding in query

Patch by Jun Omae.

Fixes #10135.

comment:7 Changed 4 months ago by Ryan J Ollos

Resolution: fixed
Status: closedreopened

comment:8 Changed 4 months ago by Ryan J Ollos

One test failure:

$python -m unittest estimationtools.tests.burndownchart.BurndownChartTestCase.test_url_encode
E
======================================================================
ERROR: test_url_encode (estimationtools.tests.burndownchart.BurndownChartTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "estimationtools/tests/burndownchart.py", line 246, in test_url_encode
    % (start, end))
  File "estimationtools/burndownchart.py", line 90, in expand_macro
    timetable = self._calculate_timetable(options, query_args, req)
  File "estimationtools/burndownchart.py", line 189, in _calculate_timetable
    tickets = execute_query(self.env, req, query_args)
  File "estimationtools/utils.py", line 151, in execute_query
    query = Query.from_string(env, query_string)
  File "/Users/rjollos/Documents/Workspace/trac-dev/teo-rjollos.git/trac/ticket/query.py", line 170, in from_string
    raise QuerySyntaxError(_('Query filter requires field and '
QuerySyntaxError: Query filter requires field and constraints separated by a "="

----------------------------------------------------------------------
Ran 1 test in 0.022s

FAILED (errors=1)

query_args is:

{'max': 0, u'hours_remaining!': None, 'milestone': 'One & Two'}

query_string is:

max=0&hours_remaining!=None&milestone=One & Two

Issue seems to be associated with estimationtoolsplugin/trunk/estimationtools/burndownchart.py@16592:188#L177.

comment:9 Changed 4 months ago by Jun Omae

Hmm. That patch is from trac:source:/tags/trac-1.0.15/trac/ticket/query.py@:1371-1372#L1366.

It seems meta characters &=% should be quoted.

  • estimationtoolsplugin/trunk/estimationtools/utils.py

    diff --git a/estimationtoolsplugin/trunk/estimationtools/utils.py b/estimationtoolsplugin/trunk/estimationtools/utils.py
    index b167c6be6..afcca8d55 100644
    a b def parse_options(env, content, options): 
    145145
    146146def execute_query(env, req, query_args):
    147147    # set maximum number of returned tickets to 0 to get all tickets at once
     148    def quote(v):
     149        return unicode(v).replace('%', '%25').replace('&', '%26') \
     150                         .replace('=', '%3D')
     151
    148152    query_args['max'] = 0
    149     query_string = '&'.join('%s=%s' % item for item in query_args.iteritems())
     153    query_string = '&'.join('%s=%s' % (quote(item[0]), quote(item[1]))
     154                            for item in query_args.iteritems())
    150155    env.log.debug("query_string: %s", query_string)
    151156    query = Query.from_string(env, query_string)
    152157
Last edited 4 months ago by Jun Omae (previous) (diff)

comment:10 Changed 4 months ago by Ryan J Ollos

Thanks, test passing now.

comment:11 Changed 4 months ago by Ryan J Ollos

Resolution: fixed
Status: reopenedclosed

In 16740:

TracEstimationTools 0.5.0dev: Quote meta characters

Fixes #10135.

Modify Ticket

Change Properties
Set your email in Preferences
Action
as closed The owner will remain Joachim Hoessler.
The resolution will be deleted.

Add Comment


E-mail address and name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.