Issue 30962  —   TXT U2.0: cannot open text file without BOM when $DEF_CHARSET=UTF8

Status:   Solved in 10.2.01

Solution available in patch(es):      G102

Description:

 Summary:
 Uniface: It is not possible to open a text file without a Unicode BOM
 (Byte-Order-Mark) when using the TXT U2.0 connector when $DEF_CHARSET is set to
 UTF8. This can cause compatibility problems, especially on Unix platforms.

 Environment:
 +Uniface: Version 9.6.03 - MX02 (and higher)
 +Operating System: OS independent
 +Database: TXT Connector U2.0
 +Additional: $DEF_CHARSET=UTF8

 Symptoms:
 It is not possible to open a text file without a Unicode BOM when using the TXT
 U2.0 connector when $DEF_CHARSET is set to UTF8. This can cause compatibility
 problems, especially on Unix platforms.

 When trying to open a text file without a Unicode BOM and $DEF_CHARSET is set
 to UTF8, the error -40 (File contents with Unicode BOM detected, cannot append
 Unicode) is returned. On the other hand it is no problem to open such a text
 file when using the TXT U1.0 connector.

Workaround:

 The workaround is to first add a BOM. This can be done by opening the relevant
 files in a text editor such as notepad.exe, going to File->Save As, choosing
 UTF-8 as its encoding and then Save.
 
 IMPORTANT - Only apply this workaround for files containing only ASCII
 characters: non-ASCII characters are translated from ANSI to UTF-8, this may
 corrupt characters that were already in UTF-8.

Notes:

 A new parameter is introduced for the TXT(2.0) connector:
   [DRIVER_SETTINGS]
   USYS$TXT_PARAMS bom=yes|no|require

 This parameter can also be set as a flag on specific TXT data files in the
 assignment file, similar to the /uni flag. Example:
   [ENTITIES]
   ent.model   $TXT:ent.ffd/append/uni=utf8/bom=yes

 It is effective only when Unicode is used. i.e. when the uni=... option is also
 used in USYS$TXT_PARAMS, when $DEF_CHARSET is set to UTF8 and/or when the
 /uni=... flag is added to specific TXT data files in the [ENTITIES] section of
 the assignment file.
 The default is 'require'. The meaning of the parameter values is as follows:

 * yes:
   When a new data file is created it will be created with a Unicode BOM.
   When reading an existing data file it is not required that it has a BOM.

 * no:
   When a new data file is created it will be created without a Unicode BOM.
   When reading an existing data file it is not required that it has a BOM.

 * require: (may be abbreviated to 'req')
   When a new data file is created it will be created with Unicode BOM.
   When reading an existing data file it is required that it has a BOM,
   if it does not have a BOM an error is issued. This is the default.