Modify

Opened 6 years ago

Closed 11 months ago

#10135 closed defect (fixed)

Umlauts doesn't work

Reported by: falkb Owned by: Joachim Hoessler
Priority: normal Component: EstimationToolsPlugin
Severity: normal Keywords:
Cc: Ryan J Ollos, osimons Trac Release: 0.12

Description

In function def execute_query(env, req, query_args): of utils.py, I tried to add

  • estimationtoolsplugin/trunk/estimationtools/utils.py

     
    134137                        .replace('+', ' ')\
    135138                        .replace('%23', '#')\
    136139                        .replace('%28', '(')\
    137                         .replace('%29', ')')
     140                        .replace('%29', ')')\
     141                        .replace('%C3%84', 'Ä')\
     142                        .replace('%C3%96', 'Ö')\
     143                        .replace('%C3%9C', 'Ü')\
     144                        .replace('%C3%A4', 'ä')\
     145                        .replace('%C3%B6', 'ö')\
     146                        .replace('%C3%BC', 'ü')\
     147                        .replace('%C3%9F', 'ß')
    138148    env.log.debug("query_string: %s" % query_string)
    139149    query = Query.from_string(env, query_string)

but just get this error:

Error: Macro WorkloadChart(milestone=testgehäuse) failed

You must not use 8-bit bytestrings unless you use a text_factory that can interpret 8-bit bytestrings (like text_factory = str). It is highly recommended that you instead just switch your application to Unicode strings.

How must umlauts be fixed?

Attachments (0)

Change History (11)

comment:1 Changed 12 months ago by anonymous

Cc: rjollos,osimonsrjollos, osimons

It's still not in the trunk, why?

comment:2 Changed 12 months ago by Jun Omae

That patch cannot fix this issue. The same issue would be raised if other unicode characters are used.

Could you please try the following patch?

  • estimationtoolsplugin/trunk/estimationtools/utils.py

    diff --git a/estimationtoolsplugin/trunk/estimationtools/utils.py b/estimationtoolsplugin/trunk/estimationtools/utils.py
    index 76e0d8bf1..b167c6be6 100644
    a b from time import strptime 
    1414from trac.config import Option, ListOption, BoolOption
    1515from trac.core import TracError, Component, implements
    1616from trac.ticket.query import Query
    17 from trac.util.datefmt import from_utimestamp, utc
    18 from trac.util.text import unicode_urlencode
     17from trac.util.datefmt import from_utimestamp
    1918from trac.web.api import IRequestHandler, RequestDone
    2019from trac.wiki.api import parse_args
    2120
    def parse_options(env, content, options): 
    147146def execute_query(env, req, query_args):
    148147    # set maximum number of returned tickets to 0 to get all tickets at once
    149148    query_args['max'] = 0
    150     # urlencode the args, converting back a few vital exceptions:
    151     # see the authorized fields in the query language in
    152     # http://trac.edgewall.org/wiki/TracQuery#QueryLanguage
    153     query_string = unicode_urlencode(query_args).replace('%21=', '!=') \
    154         .replace('%21%7E=', '!~=') \
    155         .replace('%7E=', '~=') \
    156         .replace('%5E=', '^=') \
    157         .replace('%24=', '$=') \
    158         .replace('%21%5E=', '!^=') \
    159         .replace('%21%24=', '!$=') \
    160         .replace('%7C', '|') \
    161         .replace('+', ' ') \
    162         .replace('%23', '#') \
    163         .replace('%28', '(') \
    164         .replace('%29', ')')
     149    query_string = '&'.join('%s=%s' % item for item in query_args.iteritems())
    165150    env.log.debug("query_string: %s", query_string)
    166151    query = Query.from_string(env, query_string)
    167152

comment:3 Changed 12 months ago by falkb

Jun, your patch works well. Thanks!

comment:4 Changed 12 months ago by anonymous

Could you commit the patch, please! And close this ticket then.

comment:5 Changed 11 months ago by falkb

I tried to commit the patch above but have no permissions. So I'm giving up with letting you know this could have been fixed. Neither can I set Jun Omae on CC of this ticket.

comment:6 Changed 11 months ago by Ryan J Ollos

Resolution: fixed
Status: newclosed

In 16736:

EstimationTools 0.5.0dev: Fix incorrect encoding in query

Patch by Jun Omae.

Fixes #10135.

comment:7 Changed 11 months ago by Ryan J Ollos

Resolution: fixed
Status: closedreopened

comment:8 Changed 11 months ago by Ryan J Ollos

One test failure:

$python -m unittest estimationtools.tests.burndownchart.BurndownChartTestCase.test_url_encode
E
======================================================================
ERROR: test_url_encode (estimationtools.tests.burndownchart.BurndownChartTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "estimationtools/tests/burndownchart.py", line 246, in test_url_encode
    % (start, end))
  File "estimationtools/burndownchart.py", line 90, in expand_macro
    timetable = self._calculate_timetable(options, query_args, req)
  File "estimationtools/burndownchart.py", line 189, in _calculate_timetable
    tickets = execute_query(self.env, req, query_args)
  File "estimationtools/utils.py", line 151, in execute_query
    query = Query.from_string(env, query_string)
  File "/Users/rjollos/Documents/Workspace/trac-dev/teo-rjollos.git/trac/ticket/query.py", line 170, in from_string
    raise QuerySyntaxError(_('Query filter requires field and '
QuerySyntaxError: Query filter requires field and constraints separated by a "="

----------------------------------------------------------------------
Ran 1 test in 0.022s

FAILED (errors=1)

query_args is:

{'max': 0, u'hours_remaining!': None, 'milestone': 'One & Two'}

query_string is:

max=0&hours_remaining!=None&milestone=One & Two

Issue seems to be associated with estimationtoolsplugin/trunk/estimationtools/burndownchart.py@16592:188#L177.

comment:9 Changed 11 months ago by Jun Omae

Hmm. That patch is from trac:source:/tags/trac-1.0.15/trac/ticket/query.py@:1371-1372#L1366.

It seems meta characters &=% should be quoted.

  • estimationtoolsplugin/trunk/estimationtools/utils.py

    diff --git a/estimationtoolsplugin/trunk/estimationtools/utils.py b/estimationtoolsplugin/trunk/estimationtools/utils.py
    index b167c6be6..afcca8d55 100644
    a b def parse_options(env, content, options): 
    145145
    146146def execute_query(env, req, query_args):
    147147    # set maximum number of returned tickets to 0 to get all tickets at once
     148    def quote(v):
     149        return unicode(v).replace('%', '%25').replace('&', '%26') \
     150                         .replace('=', '%3D')
     151
    148152    query_args['max'] = 0
    149     query_string = '&'.join('%s=%s' % item for item in query_args.iteritems())
     153    query_string = '&'.join('%s=%s' % (quote(item[0]), quote(item[1]))
     154                            for item in query_args.iteritems())
    150155    env.log.debug("query_string: %s", query_string)
    151156    query = Query.from_string(env, query_string)
    152157
Last edited 11 months ago by Jun Omae (previous) (diff)

comment:10 Changed 11 months ago by Ryan J Ollos

Thanks, test passing now.

comment:11 Changed 11 months ago by Ryan J Ollos

Resolution: fixed
Status: reopenedclosed

In 16740:

TracEstimationTools 0.5.0dev: Quote meta characters

Fixes #10135.

Modify Ticket

Change Properties
Set your email in Preferences
Action
as closed The owner will remain Joachim Hoessler.
The resolution will be deleted.

Add Comment


E-mail address and name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.