Re: [ANNOUNCE] Kconfiglib menuconfig implementation

[Ulf Magnusson]

On Mon, May 14, 2018 at 6:58 PM, Pavel Machek <pavel@xxxxxx> wrote:
> Hi!
>
>> > config CAN_PEAK_PCIEFD
>> >         depends on PCI
>> >         tristate "PEAK-System PCAN-PCIe FD cards"
>> >         ---help---
>> >           This driver adds support for the PEAK-System PCI Express FD
>> >           CAN-FD cards family.
>> >           These 1x or 2x CAN-FD channels cards offer CAN 2.0 a/b as well as
>> >           CAN-FD access to the CAN bus. Besides the nominal bitrate of up to
>> >           1 Mbit/s, the data bytes of CAN-FD frames can be transmitted with
>> >           up to 12 Mbit/s. A galvanic isolation of the CAN ports protects the
>> >           electronics of the card and the respective computer against
>> >           disturbances of up to 500 Volts. The PCAN-PCI Express FD can be
>> >           operated with ambient temperatures in a range of -40 to +85 ÂC.
>> >
>> >
>> > kconfiglib.KconfigSyntaxError:
>> > Malformed ascii in drivers/net/can/peak_canfd/Kconfig
>> > Context: b't temperatures in a range of -40 to +85 \xc2\xb0C.\n'
>> > Problematic data: b'\xc2'
>> > Reason: ordinal not in range(128)
>> >
>> >
>> > thanks,
>> > --
>> > ~Randy
>>
>> Thanks for trying it out!
>>
>> You're probably running in the C locale, which implies an ASCII
>> encoding. That has caused enough trouble that the Python devs decided
>> to automatically convert it to UTF-8 in Python 3.7:
>> https://www.python.org/dev/peps/pep-0538/. LC_CTYPE=C.UTF-8 should fix
>> it.
>>
>> It's a bit silly to have it crash for something like that though. I
>> could force UTF-8 instead of respecting the locale (though it feels
>> neater to respect settings), or tell Python to ignore decoding errors.
>> Should probably do something at least...
>
> Well, Kconfig files are utf-8, as is the rest of kernel.

Yeah, it's the only sane assumption anyway.

>
> So yes, you should respect $LC_CTYPE for display etc, but you should always
> assume Kconfig files are utf-8... and that does not need to be optional.

Already made it optional, with UTF-8 as the default (via
Kconfig.__init__(..., encoding="utf-8")).

I did a similar thing to what PEP 538 does for the menuconfig:
https://github.com/ulfalizer/Kconfiglib/blob/master/menuconfig.py#L2271.
The C locale is converted to a UTF-8 locale for LC_CTYPE if possible.
Other locales are respected. That'll be the default behavior in Python
3.7, and prevents issues with get_wch().

>
>                                                                         Pavel

Cheers,
Ulf