Author Topic: Using OxygenBasic with Unicode  (Read 1258 times)

0 Members and 1 Guest are viewing this topic.

Charles Pegge

  • Admin Support Member
  • *****
  • Posts: 3751
    • Oxygen Basic
Re: Using OxygenBasic with Unicode
« Reply #15 on: July 01, 2018, 07:26:45 AM »
Hi Roland,

Using _wfopen from msvcrt.inc.

Code: [Select]
uses corewin

function wGetFile(wstring name) as wstring
==========================================
sys f
int e
wstring m="rb"
bstring2 s
f=_wfopen name,m      'open for reading binary
fseek f,0,2           'end of file
e=ftell f             'get position
fseek f,0,0           'beginning of file
strptr s=getmemory e  'create buffer to fit
fread s,1,e,f         'load buffer
fclose f              'close file
return s
end function


function wPutFile(wstring name, wstring s) as int
=================================================
sys f
wstring m="wb"
int e=len s
f=_wfopen name,m      'open for writing binary
e=fwrite s,1,e*2,f    'save data
fclose f              'close file
return e
end function
'
'TESTS
======
'
wstring s=wgetfile "Multilingual.txt"
print s
wputfile "t.txt",s

Arnold

  • Hero Member
  • *****
  • Posts: 677
Re: Using OxygenBasic with Unicode
« Reply #16 on: July 01, 2018, 11:17:32 AM »
Hi Charles,

thank you for the help. I think the functions are useful. By applying them I noticed that wGetFile will crash if I use an invalid filename. I inserted this line:
if f=0 then s="" : return s

but I do not know if this is sufficient; I do not know the effect on bstring2 s.

I also found that I must also load / save strings, so I added overloaded functions. Probably they can be combined with a macro, but I am not very good in this. So this would be my temporary solution:

Code: [Select]
$ filename "UnicodeFilename.exe"
'uses rtl32
'uses rtl64

uses corewin

function wGetFile(wstring name) as wstring
==========================================
sys f
int e
wstring m="rb"
bstring2 s
f=_wfopen name,m      'open for reading binary
if f=0 then s="" : return s
fseek f,0,2           'end of file
e=ftell f             'get position
fseek f,0,0           'beginning of file
strptr s=getmemory e  'create buffer to fit
fread s,1,e,f         'load buffer
fclose f              'close file
return s
end function

function wGetFile(wstring name) as string
==========================================
sys f
int e
wstring m="rb"
bstring s
f=_wfopen name,m      'open for reading binary
if f=0 then s="" : return s
fseek f,0,2           'end of file
e=ftell f             'get position
fseek f,0,0           'beginning of file
strptr s=getmemory e  'create buffer to fit
fread s,1,e,f         'load buffer
fclose f              'close file
return s
end function


function wPutFile(wstring name, wstring s) as int
=================================================
sys f
wstring m="wb"
int e=len s
f=_wfopen name,m      'open for writing binary
e=fwrite s,1,e*2,f    'save data
fclose f              'close file
return e
end function

function wPutFile(wstring name, string s) as int
=================================================
sys f
wstring m="wb"
int e=len s
f=_wfopen name,m      'open for writing binary
e=fwrite s,1,e,f      'save data
fclose f              'close file
return e
end function

'
'TESTS
======

string fname="multilingual.txt"
wstring s= (wstring) wgetfile(fname)
print len s
print s
wputfile "t1.txt",s

wstring fn="multilingual.o2bas"
string txt= wgetfile(fn)
print len txt
print txt

wputfile "t2.txt",txt

Charles Pegge

  • Admin Support Member
  • *****
  • Posts: 3751
    • Oxygen Basic
Re: Using OxygenBasic with Unicode
« Reply #17 on: July 02, 2018, 01:38:31 PM »
Hi Roland,

This has macros to select Unicode file i/o on the basis of the data string type using widthof()

Code: [Select]
uses corewin

function wGetFile(wstring n) as wstring
======================================
sys f
int e
wstring m="rb"
bstring2 s
f=_wfopen n,m         'open for reading binary
if not f then return ""
fseek f,0,2           'end of file
e=ftell f             'get position
fseek f,0,0           'beginning of file
strptr s=getmemory e  'create buffer to fit
fread s,1,e,f         'load buffer
fclose f              'close file
return s
end function


function wGetFile(wstring n,*s)
==============================
s=wGetFile n
end function


function wPutFile(wstring n,s) as int
====================================
sys f
wstring m="wb"
int e=len s
f=_wfopen n,m         'open for writing binary
if not f then return 0
e=fwrite s,1,e*2,f    'save data
fclose f              'close file
return e
end function
'
macro ugetfile (f,s)
===================
#if widthof(s)=2
  wgetfile(f,s)
#else
  getfile(f,s)
#endif
end macro
'
macro uputfile (f,s)
===================
#if widthof(s)=2
  wputfile(f,s)
#else
  putfile(f,s)
#endif
end macro
'
'
'TESTS
======
'
wstring f="t.txt"
wstring s
's=getfile f
ugetfile(f,s)
print s
f="s.txt"
uputfile(f,s)

Arnold

  • Hero Member
  • *****
  • Posts: 677
Re: Using OxygenBasic with Unicode
« Reply #18 on: July 03, 2018, 12:13:51 AM »
Hi Charles,

these are my tests:

string fname="multilingual.txt"
wstring s= getfile(fname)
ugetfile(fname,s)
print len s
print s
uputfile ("t1.txt",s)

wstring fn="multilingual.o2bas"
wstring txt
txt=getfile(fn)
ugetfile(fn,txt)
print len txt
'print txt
uputfile ("t2.txt",txt)

print txt displays the string as unicode chars, but t2.txt is saved correctly. I certainly can use any of the approaches above for my REdit experiments.

By mistake in the beginning I used class "Richedit20A" which makes a difference if I use RegisterClassW for an unicode window. If I load an ansi text I must translate it to unicode.
Currently I can open ansi, UTF-8, UTF-16 files. Right now, I am investigating if it is possible to display an RTF file too. Otherwise I will omit this option.

Roland

Edit: it seems I can open and display .rtf files in every mode. I am a bit confused.

« Last Edit: July 03, 2018, 01:42:42 AM by Arnold »

Arnold

  • Hero Member
  • *****
  • Posts: 677
Re: Using OxygenBasic with Unicode
« Reply #19 on: July 03, 2018, 08:27:33 AM »
Hi Charles,

this is my result for creating a richedit control using the unicode functions. This is not intended to be an editor, only to show the different effects on loading a file as ansi, utf16, utf8, rtf or plain text. I used your initial wGetFile method, I did not test the later version.

Attached is a zip file with the source code and three testfiles in different formats, but the app can be tested with any type of text.

This will end my basic tests with unicode. I assume all the necessary tools are available to start unicode programming seriously?

Roland

Edit: Although I did not add a right-click popup menu, it should be possible to use Ctrl-C / copy selected text, Ctrl-V / paste text into RichEdit, Ctrl-Z / Undo and other short-cuts.

Code: OxygenBasic
  1. 'https://docs.microsoft.com/de-de/windows/desktop/Controls/rich-edit-controls
  2.  
  3. $ filename "RicheditUnicode.exe"
  4.   'uses RTL32
  5.  'uses RTL64
  6.  
  7.    uses WinData
  8.  
  9.    % COLOR_WINDOW = 5
  10.    % ES_SAVESEL=0x8000
  11.    % SWP_NOZORDER=4
  12.    % SS_LEFT=0
  13.    % SS_NOTIFY=0x0100  
  14.    % SF_TEXT=1
  15.    % EM_SETTEXTEX=1121
  16.    % ST_UNICODE=8
  17.    % CP_ACP=0
  18.    % CP_UTF8=65001 'codepage UTF8
  19.   % WM_CLEAR=771  
  20.  
  21.    type SETTEXTEX
  22.      dword flags
  23.      uint  codepage
  24.    end type
  25.    
  26.  
  27.    extern lib "Kernel32.dll"
  28.    ! GetCommandLine         "GetCommandLineW"      '0
  29.   ! GetModuleHandle        "GetModuleHandleW"     '1
  30.   end extern
  31.  
  32.    extern lib "User32.dll"
  33.    ! CreateWindowEx        "CreateWindowExW"       '12
  34.   ! DefWindowProc         "DefWindowProcW"        '4
  35.   ! DispatchMessage       "DispatchMessageW"      '1
  36.   ! GetClientRect                                 '2
  37.   ! GetMessage            "GetMessageW"           '4
  38.   ! GetSystemMetrics                              '1
  39.   ! IsWindowUnicode
  40.    ! LoadIcon              "LoadIconW"             '2
  41.   ! LoadCursor            "LoadCursorW"           '2
  42.   ! MessageBox            "MessageBoxW"           '1
  43.   ! PostQuitMessage                               '1
  44.   ! RegisterClass         "RegisterClassW"        '1                                
  45.   ! SendMessage           "SendMessageW"          '4
  46.   ! SetFocus
  47.    ! SetWindowPos                                  '7
  48.   ! SetWindowText         "SetWindowTextW"        '2  
  49.   ! ShowWindow                                    '2
  50.   ! TranslateMessage                              '1
  51.   ! UpdateWindow                                  '1
  52.   end extern
  53.  
  54.    extern lib "Comctl32.dll"
  55.    ! InitCommonControlsEx                          '1
  56.   end extern
  57.  
  58.    #ifndef mode64bit
  59.      extern lib "Msvcrt.dll" cdecl
  60.    #else
  61.      extern lib "Msvcrt.dll"
  62.    #endif
  63.    ! _wfopen
  64.    ! fclose
  65.    ! fread
  66.    ! fseek
  67.    ! ftell
  68.    end extern
  69.  
  70.    function wGetFile(wstring name) as wstring
  71.    ==========================================
  72.    sys f
  73.    int e
  74.    wstring m="rb"
  75.    bstring2 s
  76.    f=_wfopen name,m      'open for reading binary
  77.   if f=0 then return ""  
  78.    fseek f,0,2           'end of file
  79.   e=ftell f             'get position
  80.   fseek f,0,0           'beginning of file
  81.   strptr s=getmemory e  'create buffer to fit
  82.   fread s,1,e,f         'load buffer
  83.   fclose f              'close file
  84.   return s
  85.    end function
  86.  
  87.    function wGetFile(wstring name) as string
  88.    ==========================================
  89.    sys f
  90.    int e
  91.    wstring m="rb"
  92.    bstring s
  93.    f=_wfopen name,m      'open for reading binary
  94.   if f=0 then return ""
  95.    fseek f,0,2           'end of file
  96.   e=ftell f             'get position
  97.   fseek f,0,0           'beginning of file
  98.   strptr s=getmemory e  'create buffer to fit
  99.   fread s,1,e,f         'load buffer
  100.   fclose f              'close file
  101.   return s
  102.    end function
  103.  
  104.    'create a structure of INITCOMMONCONTROLSEX
  105.   INITCOMMONCONTROLSEXt iccex
  106.    
  107.    iccex.dwSize=sizeof(iccex)
  108.    'Register Common Controls
  109.   iccex.dwICC= 0xffff
  110.    InitCommonControlsEx(@iccex)
  111.      
  112.    LoadLibrary("RICHED20.DLL")
  113.  
  114.  
  115.    declare function WinMain(sys inst, prevInst, asciiz2*cmdline, sys show) as sys
  116.  
  117.    '=========
  118.   'MAIN CODE
  119.   '=========
  120.  
  121.    dim cmdline as wchar ptr, hInstance as sys
  122.    @cmdline=GetCommandLine
  123.    hInstance=GetModuleHandle 0
  124.  
  125.    sys hButton1, hButton2, hREdit, hEdit, Group
  126.    % ID_BUTTON1=1000
  127.    % ID_BUTTON2=1001
  128.    sys lbl[5]
  129.    wstring lblTxt[4]={"   Ansi","UTF-16","UTF-8","Text","Open file as: "}
  130.    sys radio[4]
  131.    sys id_radio[4]={101,102,103,104}
  132.  
  133.  
  134.    'WINDOWS START
  135.   '=============
  136.  
  137.    WinMain hInstance,0,cmdline,SW_NORMAL
  138.    end
  139.  
  140.  
  141. function WinMain(sys inst, prevInst, asciiz2*cmdline, sys show) as sys
  142.  
  143.    WndClass wc
  144.    MSG      wm
  145.  
  146.    sys hwnd, Wwd, Wht, Wtx, Wty, Tax
  147.    wstring classname="REDemo"
  148.  
  149.    wc.style = CS_HREDRAW or CS_VREDRAW
  150.    wc.lpfnWndProc = @WndProc
  151.    wc.cbClsExtra =0
  152.    wc.cbWndExtra =0
  153.    wc.hInstance =inst
  154.    wc.hIcon=LoadIcon 0, IDI_APPLICATION
  155.    wc.hCursor=LoadCursor 0,IDC_ARROW
  156.    wc.hbrBackground = COLOR_WINDOW
  157.    wc.lpszMenuName =null
  158.    wc.lpszClassName = strptr classname
  159.  
  160.    RegisterClass (@wc)
  161.  
  162.    Wwd = 640 : Wht = 400
  163.    Tax = GetSystemMetrics SM_CXSCREEN
  164.    Wtx = (Tax - Wwd) /2
  165.    Tax = GetSystemMetrics SM_CYSCREEN
  166.    Wty = (Tax - Wht) /2
  167.  
  168.    'Main Window
  169.   hwnd = CreateWindowEx 0,wc.lpszClassName,  wstring("OXYGEN BASIC"),WS_OVERLAPPEDWINDOW,Wtx,Wty,Wwd,Wht,0,0,inst,0
  170.  
  171.    'Buttons
  172.   hButton1=CreateWindowEx(0,
  173.       wstring("Button"), wstring("Open File"),
  174.       WS_CHILD | WS_VISIBLE | BS_PUSHBUTTON,
  175.       500, 10, 80 ,25,
  176.       hwnd,ID_BUTTON1,inst, 0)
  177.    hButton2=CreateWindowEx(0,
  178.       wstring("Button"), wstring("Clear REdit"),
  179.       WS_CHILD | WS_VISIBLE | BS_PUSHBUTTON,
  180.       500, 45, 80 ,25,
  181.       hwnd,ID_BUTTON2,inst, 0)
  182.  
  183.    'Edit
  184.   hEdit=CreateWindowEx(WS_EX_CLIENTEDGE,
  185.       wstring("Edit"), wstring(""),
  186.       WS_CHILD | WS_VISIBLE | ES_LEFT | WS_BORDER | ES_AUTOHSCROLL,
  187.       20, 10, 450 ,25,
  188.       hwnd,0,inst,0)
  189.    
  190.    'Labels
  191.   int x, lft, wid
  192.    lft=130 : wid=50
  193.    for x=1 to 5
  194.       if x=5 then lft=-340 : wid=90
  195.       lbl[x]=CreateWindowEx(0,
  196.       wstring("Static"), wstring(""),
  197.       WS_CHILD | WS_VISIBLE | SS_LEFT,
  198.       lft+((x-1)*90), 45, wid ,25,
  199.       hwnd,0,inst,0)
  200.       SetWindowText(lbl[x],lblTxt[x])
  201.    next x
  202.  
  203.    'Radios
  204.   for x=1 to 4
  205.       radio[x]=CreateWindowEx(0,
  206.       wstring("Button"), wstring(""),
  207.       WS_CHILD | WS_VISIBLE | BS_AUTORADIOBUTTON,
  208.       180+((x-1)*90), 45, 15 ,15,
  209.       hwnd,id_radio[x],inst,0)
  210.    next x
  211.    SendMessage(radio[1], BM_SETCHECK, true,0)
  212.    
  213.    'Richedit
  214.   hREdit=CreateWindowEx(WS_EX_CLIENTEDGE,
  215.       wstring("RichEdit20W"), wstring(""),
  216.       WS_CHILDWINDOW|WS_VISIBLE | WS_VSCROLL | WS_HSCROLL | ES_AUTOVSCROLL | ES_SAVESEL | ES_MULTILINE | WS_BORDER | ES_WANTRETURN,
  217.       20, 60,200,25,
  218.       hwnd,0,inst,0)
  219.  
  220.    SetFocus(hEdit)
  221.    ShowWindow hwnd,SW_SHOW
  222.    UpdateWindow hwnd
  223.  
  224.    sys bRet
  225.    do while bRet := GetMessage (@wm, 0, 0, 0)
  226.      if bRet = -1 then
  227.        'show an error message
  228.     else
  229.        TranslateMessage @wm
  230.        DispatchMessage @wm
  231.      end if
  232.    wend
  233.  
  234. end function
  235.  
  236.  
  237. function WndProc (sys hWnd, uint wMsg, sys wParam, lparam ) as sys callback
  238.    SETTEXTEX settext
  239.  
  240.    select wMsg
  241.  
  242.       case WM_COMMAND
  243.         select loword(wParam)        
  244.            case ID_BUTTON1
  245.               wstring fName = space 256
  246.               SendMessage(hEdit,  WM_GETTEXT, len(fName), fName)
  247.               fname=ltrim(rtrim(fname))              
  248.               if len(fname)>0 then
  249.                 if SendMessage(Radio[1],BM_GETCHECK,0,0)=BST_CHECKED then 'Ansi
  250.                  string text1
  251.                   text1 = wgetfile (fName)                              
  252.                   if len(text1)=0 then
  253.                     MessageBox(hWnd, wstring(fName) + " missing or load failure!", wstring("Load File"), MB_OK or MB_ICONASTERISK)
  254.                   else                
  255.                     'RegisterClassW creates Unicode window, so translate to Unicode
  256.                    settext.flags=ST_UNICODE
  257.                     settext.codepage=CP_ACP  'System codepage
  258.                    SendMessage(hREdit, EM_SETTEXTEX, &settext, text1)
  259.                   end if
  260.                 end if  
  261.  
  262.                 if SendMessage(Radio[2],BM_GETCHECK,0,0)=BST_CHECKED then 'Unicode
  263.                  wstring text2
  264.                   text2 = (wstring) wgetfile (fName)                                
  265.                   if len(text2)=0 then
  266.                     MessageBox(hWnd, wstring(fName) + " load failure!", wstring("Load File"), MB_OK or MB_ICONASTERISK)
  267.                   else
  268.                     SendMessage(hREdit, WM_SETTEXT, 0, text2)
  269.                   end if
  270.                 end if  
  271.  
  272.                 if SendMessage(Radio[3],BM_GETCHECK,0,0)=BST_CHECKED then 'UTF8
  273.                  string text3
  274.                   wstring text3 = (wstring) wgetfile (fName)                              
  275.                   if len(text3)=0 then
  276.                     MessageBox(hWnd, wstring(fName) + " missing or load failure!", wstring("Load File"), MB_OK or MB_ICONASTERISK)
  277.                   else                
  278.                     'RegisterClassW creates Unicode window, so translate to Unicode
  279.                    settext.flags=ST_UNICODE
  280.                     settext.codepage=CP_UTF8  'cp 65001
  281.                    SendMessage(hREdit, EM_SETTEXTEX, &settext, text3)
  282.                   end if
  283.                 end if  
  284.  
  285.                 if SendMessage(Radio[4],BM_GETCHECK,0,0)=BST_CHECKED then 'Plain Text
  286.                  string text4
  287.                   text4 = wgetfile (fName)                              
  288.                   if len(text4)=0 then
  289.                     MessageBox(hWnd, wstring(fName) + " missing or load failure!", wstring("Load File"), MB_OK or MB_ICONASTERISK)
  290.                   else                
  291.                     'Workaround
  292.                    text4=chr(32) & text4
  293.                     SendMessage(hREdit, WM_SETTEXT, SF_TEXT, text4)
  294.                   end if
  295.                 end if  
  296.  
  297.               end if
  298.              
  299.            case ID_BUTTON2
  300.               SendMessage(hREdit, EM_SETSEL, 0, -1)
  301.               SendMessage(hREdit, WM_CLEAR, 0, 0)                              
  302.         end select
  303.  
  304.       case WM_SIZE      
  305.          RECT rcClient
  306.      
  307.          // Calculate remaining height and size edit
  308.          GetClientRect(hwnd, &rcClient)
  309.          SetWindowPos(hREdit, NULL, 0, rcClient.top+75, rcClient.right, rcClient.bottom-75, SWP_NOZORDER)
  310.            
  311.       case WM_DESTROY
  312.          PostQuitMessage 0
  313.  
  314.       case WM_KEYDOWN
  315.          Select wParam
  316.             Case 27 : SendMessage hwnd, WM_CLOSE, 0, 0      'ESCAPE
  317.         End Select
  318.  
  319.       case else
  320.          function=DefWindowProc hWnd,wMsg,wParam,lParam
  321.  
  322.    end select
  323.  
  324. end function ' WndProc
  325.  
« Last Edit: July 03, 2018, 10:01:40 AM by Arnold »

Arnold

  • Hero Member
  • *****
  • Posts: 677
Re: Using OxygenBasic with Unicode
« Reply #20 on: July 04, 2018, 10:55:15 AM »
When I started searching in Internet for unicode examples, by chance I found this site in Josť Roca's forum:
http://www.jose.it-berater.org/smfforum/index.php?topic=5253.msg22686#msg22686

I came to the same conclusion. Unicode is a world of its own. Therefore this would be the idea:

create a separate directory (OxygenUnicode?), copy oxygen.dll, co2.exe, \inc\rtl32.inc, \inc\rtl64.inc into the folder and sub folder, create folder \examples and start converting the examples which can be done using unicode. This will need some time. Nothing has to be changed with the original installation and there is no need for #ifdef unicode. But the differences between ansi and unicode could be elaborated much easier. The implementation of the include files in the \inc subfolder would mostly be unicode specific.

Would this approach make sense?
« Last Edit: July 04, 2018, 10:38:36 PM by Arnold »

John

  • Hero Member
  • *****
  • Posts: 3280
Re: Using OxygenBasic with Unicode
« Reply #21 on: July 04, 2018, 11:30:07 AM »
I like thinBasic's approach with a core and extended with modules.

Charles Pegge

  • Admin Support Member
  • *****
  • Posts: 3751
    • Oxygen Basic
Re: Using OxygenBasic with Unicode
« Reply #22 on: July 04, 2018, 09:21:41 PM »
Many thanks for your examples, Roland.

We have quite a good collection now. I'll think about how Unicode should be integrated with the rest of the system, for instance being used in combination with Opengl and maths, as well as Windows GUI.

Arnold

  • Hero Member
  • *****
  • Posts: 677
Re: Using OxygenBasic with Unicode
« Reply #23 on: July 05, 2018, 12:12:41 AM »
Hi Charles,

my intention is not to muddle in Oxygenbasic in any way. It is clear for me that O2 can deal with unicode very well. Yet I found it is also a different way of programming.
Perhaps there are other options possible e.g.: for unicode specific include files ConsoleW.inc, MinWinW.inc, CoreWinW.inc etc. which use only the unicode functions, and an extension like .ubas/.oubas for the unicode programs. Probably there are some more options possible. But I think that the directive: #ifdef unicode which is used in some languages would increase the size of the source code and also make it a bit unclear.

Nevertheless UTF-16 (Windows) and UTF-8 (Linux, OSX) are encoding systems which are used worldwide. Without these systems some applications like translator apps would not be possible this comfortable way.

Roland

Josť Roca

  • Jr. Member
  • **
  • Posts: 87
Re: Using OxygenBasic with Unicode
« Reply #24 on: July 05, 2018, 07:43:25 AM »
Duplicating procedures and include files is an utterly waste of time. Unicode fits all, so change the code to work with unicode and forget all that useless ansi stuff forever. Windows supports the "A" functions for backward compatibility only, but almost all the "A" functions are mere wrappers that convert the string parameters to unicode and call the "W" functions. I don't know why so many people are afraid of unicode. It does not bite.

John

  • Hero Member
  • *****
  • Posts: 3280
Re: Using OxygenBasic with Unicode
« Reply #25 on: July 05, 2018, 10:35:37 AM »
Would that lock O2 into Windows only? I would like to see a flag like base index option that would default strings to unicode or ansi.

Josť Roca

  • Jr. Member
  • **
  • Posts: 87
Re: Using OxygenBasic with Unicode
« Reply #26 on: July 05, 2018, 10:57:47 AM »
If you haven't noticed it, it is already locked. It uses BSTRings and the Windows OLE engine. Even ansi strings are BSTRings allocated with SysAllocStringByteLen. Anyway, what was talking about was that using ansi instead of unicode with the Windows API is useless.
« Last Edit: July 05, 2018, 11:20:43 AM by Josť Roca »

Charles Pegge

  • Admin Support Member
  • *****
  • Posts: 3751
    • Oxygen Basic
Re: Using OxygenBasic with Unicode
« Reply #27 on: July 05, 2018, 11:35:09 AM »
Switching from ANSI to Unicode would not be difficult. But what concerns me is the schism created between physical keyboards, programming languages, symbols (dlls use ANSI) on one hand,  and the Unicode world on the other.


PS: We could carry our own bstring layer for other platforms.

John

  • Hero Member
  • *****
  • Posts: 3280
Re: Using OxygenBasic with Unicode
« Reply #28 on: July 05, 2018, 01:29:08 PM »
I personally have yet to do a project for a client where unicode was a requirement. I see multi-language support more prevalent in web programming.

Quote
Unicode contains a repertoire of over 137,000 characters covering 146 modern and historic scripts, as well as multiple symbol sets.

Quote
UTF-8 is a compromise character encoding that can be as compact as ASCII (if the file is just plain English text) but can also contain any unicode characters (with some increase in file size). UTF stands for Unicode Transformation Format. The '8' means it uses 8-bit blocks to represent a character.

Quote
PS: We could carry our own bstring layer for other platforms.

It would be great to see how far we could get running 64 bit O2 on Linux with header changes and tweaks.

Does ASM source code support Unicode text?
« Last Edit: July 07, 2018, 03:14:05 AM by John »

Charles Pegge

  • Admin Support Member
  • *****
  • Posts: 3751
    • Oxygen Basic
Re: Using OxygenBasic with Unicode
« Reply #29 on: July 07, 2018, 06:37:36 AM »
Hi John,

ASM knows nothing about strings, only bytes, words etc. but o2 string literals are currently treated as 8bit ASCII.

o2 scripts are 8bit by default, but an o2 compiler could pass UTF16 scripts after specifying o2_mode 2 (or 10 for bstrings). This would convert down to UTF8 internally.
« Last Edit: July 07, 2018, 06:46:55 AM by Charles Pegge »