Author Topic: Updating OxyScheme  (Read 19548 times)

0 Members and 1 Guest are viewing this topic.

Arnold

  • Hero Member
  • *****
  • Posts: 973
Updating OxyScheme
« on: November 02, 2019, 06:04:43 AM »
Hi Mike,

Quote
If you choose to engage yourself in OxyScheme upgrade, I'll be glad to give you a helping hand wherever I can.

although Christmas is coming up (sigh), I simply could not resist and started to remove the prototypes of mscvrt.dll. This was the easy part. I noticed that _strlwr does not work correctly unprototyped and I used lcase instead. I also used copy0 instead of strcpy. Maybe some more functions can be substituted.

At the moment my intention is to get the same results as o2scm.o2bas. I would need a little help about these issues:

In function alloc_cellseg (about line 496) I applied getmemory instead of malloc. Is there a place where I could use freememory too?

As goto can now be used globally, it should be possible to apply this statement instead of setjmp/longjmp? I assume this would be helpful for 64-bit compatibility, but at the moment I do not really know the results of longjmp. Perhaps Charles can give advice how goto can be used instead of longjmp.

There is another interesting aspect. If I apply: (load "ascii_mandel.scm") there will be an error message in about line 2355: "unable to car a non-pair element" and the app will crash. I suppose Error_0 must be checked.

Attached is OxyScheme.o2bas as a zip file. I commented the modifications with 'rs, the line numbering should almost be the same. If there are no objections, I would like to remove the prototypes of msvcrt.dll.

Roland


« Last Edit: November 05, 2019, 01:00:52 AM by Arnold »

Mike Lobanovsky

  • Hero Member
  • *****
  • Posts: 1993
Re: Updating OxyScheme
« Reply #1 on: November 02, 2019, 09:39:53 AM »
Hi Roland,

Thanks again for taking interest in my code.

0. First of all, please tell me exactly which build of Oxygen you are using for compilation so that we could have identical tool chains for our experimentation.

1. My initial code used the following convention to distinguish between the remarks (both FBSL and O2 support C-style remarks in BASIC code):
-- ANSI C remarks /*...*/ = original C code author's remarks (I am not the original author of that public domain code)
-- CPP-style remarks // = my own remarks to describe the fixes I applied while translating the code to O2
-- (occasional and incomplete) BASIC-style remarks ' = Charles' later fixes to the code introduced to maintain compatibility with later O2 builds.

What kind of remarks are we going to introduce to preserve the original convention? (I presume 'rs refers to you as 'ml can refer to me?)

2. No, AFAIK goto, either local or global, cannot substitute setjmp/longjmp. The difference is that goto jumps to its jump target with the processor registers preserved as-is at the moment the jump is taken. Conversely, setjmp preserves the contents of all processor registers in that mysterious jmp_buf[_JBLEN] structure (or array of structures if nested store/restore points are allowed) in response to the setjmp call, and then uses those values to restore automatically the processor registers in case a crash occurs somewhere in between the setjmp/longjmp calls. (each (possibly nested) setjmp should be complemented with a matching longjmp for the technique to work properly)

Setjmp/longjmp is a Linuxoid (partial) equivalent to Microsoft's full-fledged SEH (structured exception handling) mechanism -- try/except pair. The difference between the two is that, while try/except keeps track of all intermediate calls (hence, both stack and malloc memory allocations) and unwinds (i.e. brings the registers and frees malloc allocations to their initial states) all of them one by one in a nested LIFO manner, setjmp/longjmp doesn't track the intermediate call stack and thus is able to bring the processor directly to its initial state only. Which means that while the stack pointer (ESP register) can be restored faultlessly effectively nullifying all of intermediate stack allocations, possible malloc memory volumes will remain un-deallocated thus causing cumulative memory leaks.

3. There is no matching free_cellseg() to alloc_cellseg() because this particular Scheme implementation does not deallocate the total number of memory segments it accumulates while executing a particular Scheme script within a common Scheme session (i.e. before you exit the Scheme prompt altogether). It rather reuses the existing (i.e. already accumulated) amount of memory segments as needed to host the number of temp vars the script actually needs, free_cell being the "head" marker of unused cells in the thus accumulated total segment memory space.

I should also note that neither the original C code nor my DynC version holds a single call to free() while there is one in the OxyScheme implementation -- see Case OP_STRAPP in the opexe_7() function. I presume it appeared there as one of Charles' (uncommented) later upgrades to my code, probably for compatibility reasons.

4. The existing OxyScheme implementation doesn't end the current Scheme session on (quit) command but rather hangs it. It is arguably a by-product of Charles' compatibility mods in Eval_Cycle().

5. Be specifically cautious with cases that involve 64-bit integers. Note well that Oxygen's quads are in fact floating-point entities stored and processed on the FPU while Scheme's (and C language's) long integers are genuine whole integers stored and returned in the eax:edx (low 32 bits:high 32 bits) register pair.

6. I am methodologically against remming out all C functions blindly. We should rather rem them out one by one running unit tests each time to see if their O2 functional equivalents reproduce the desired compatible effect in full. Do you know Scheme enough to generate at least the simplest Scheme scripts for unit testing?
Mike
(3.6GHz Intel Core i5 Quad w/ 16GB RAM, nVidia GTX 1060Ti w/ 6GB VRAM, Windows 7 Ultimate Sp1)

Arnold

  • Hero Member
  • *****
  • Posts: 973
Re: Updating OxyScheme
« Reply #2 on: November 03, 2019, 04:11:52 AM »
Hi Mike,

thank you for your explanations. I wish I had that accuracy, too.

I always use the latest version of O2, in this case version: 0.2.8 2019-09-20T12:40:14.

After including corewin.inc I commented out the msvcrt.dll functions line by line, then always executed Oxyscheme.o2bas and O2scm.o2bas, loaded and executed the demo files to see if there is a different behaviour. So I found that there is a difference with _strlwr prototyped and unprototyped, but that I could also use lcase instead of _strlwr. I think built-in Oxygen functions should work a bit faster if they do not depend on msvcrt.dll?

Would it be okay if the comments will stay and setjmp/longjmp are not currently being tracked? So the functions could be uncommented if there is a differenc to O2scm.o2bas. But it would also be ok to comment out the protoyed funtions later.

Now I come to my weakness, which will hopefully not deter you. Although I have tried several times to better understand Scheme or Lisp, I have repeatedly failed. But I will continue until I understand the examples of Robbek and your examples. Oxyscheme (and Tinyscheme) will help me with that. Therefore, I am probably well suited to make any weird mistakes that can be made as a beginner.

At the moment simple errors like (1 + 1) will crash Oxyscheme / o2scm. I am wondering if it is possible to quickly discover the error that arises when loading ascii_mandel.scm (Error_0). Then probably already 99% of the problems are solved.

Roland

Mike Lobanovsky

  • Hero Member
  • *****
  • Posts: 1993
Re: Updating OxyScheme
« Reply #3 on: November 03, 2019, 04:59:00 AM »
My friend,

Your OxyScheme.o2bas executed by older versions of OxygenBasic (e.g. 0.1.0 2019-02-03T20:36:03) seems to load and run flawlessly. Which means the crashes are not so much OxyScheme's but rather Charles' fault.

Quote
At the moment simple errors like (1 + 1) will crash Oxyscheme / o2scm.
;D

(1 + 1) is not a valid Scheme syntax. As can be seen in the screenshot below, OxyScheme successfully recovers from such typos flagging a corresponding error, and happily executes a valid statement (+ 1 1) instead.
Mike
(3.6GHz Intel Core i5 Quad w/ 16GB RAM, nVidia GTX 1060Ti w/ 6GB VRAM, Windows 7 Ultimate Sp1)

Mike Lobanovsky

  • Hero Member
  • *****
  • Posts: 1993
Re: Updating OxyScheme
« Reply #4 on: November 03, 2019, 05:22:26 AM »
Re. setjmp/longjmp: Yes, you can rem out the respective calls throughout the script but it may make full recovery from some (not all!) errors in the user .scm script or manual input impossible and crash-prone.

Re. built-in O2 functions: No, not necessarily. Though Charles' asm code is usually of very high quality, msvcrt.dll is also an extremely well optimized and time-proven piece of programming art. If both the runtime function and its O2 equivalent are implemented as functions, then the call and execution time will be the same if both implementations are of similar complexity. DLLs are mapped into the process memory, and calls to their functions are exactly as fast as calls to process main memory.

OTOH if Charles' intrinsics are implemented as asm macros, then they may really run faster because they would avoid the call overhead.
Mike
(3.6GHz Intel Core i5 Quad w/ 16GB RAM, nVidia GTX 1060Ti w/ 6GB VRAM, Windows 7 Ultimate Sp1)

Arnold

  • Hero Member
  • *****
  • Posts: 973
Re: Updating OxyScheme
« Reply #5 on: November 03, 2019, 07:51:50 AM »
Hi Mike,

luckily I have not yet deleted version 0.1.0, and you are right: O2scm.o2bas does not crash if I enter (1 + 1), it shows the error message and I will get the chance to enter (+ 1 1). Loading ascii_mandel.scm will give an error nevertheless but the app does not crash. Now perhaps it will be easier to find out what works differently.

I personally do not mind if something does not work as expected after an update of O2. I found so many older programs e.g. developed in C which cannot be compiled without modifications when using newer versions of e.g. TCC or Pelles C. In my opinion each new version of O2 offers new benefits, and in fact some of them I would really miss.

Roland

Arnold

  • Hero Member
  • *****
  • Posts: 973
Re: Updating OxyScheme
« Reply #6 on: November 05, 2019, 01:10:59 AM »
Hi Mike,

I deleted the zip file of my first message. Probably you will not use it anyway. I was not aware that there is a different behaviour of o2scm.o2bas in version 0.1.0 and 0.2.8 of Oxygen. As long as it is not clear where the behavior deviates, it of course makes no sense to apply unprototyped functions.

Roland

Mike Lobanovsky

  • Hero Member
  • *****
  • Posts: 1993
Re: Updating OxyScheme
« Reply #7 on: November 05, 2019, 03:00:39 AM »
Hi Roland,

I see. I think Charles has been very lucky to successfully maintain the script of such complexity for such a long time, practically throughout Oxygen's entire alpha phase. Now that beta is out that isn't supposed to basically change the way language features work (API/ABI-wise) any more, the script code may be finalized to fit both 32 and 64 bits.

But to do this, the programmer has to have a clear understanding of not only how Scheme works and what C features can best fit the language needs, but also how modern beta-stage Oxygen works. Regretfully, I didn't have sufficient spare time to follow the O2 development close enough to be able to cope with such a challenge ATM.
Mike
(3.6GHz Intel Core i5 Quad w/ 16GB RAM, nVidia GTX 1060Ti w/ 6GB VRAM, Windows 7 Ultimate Sp1)

Arnold

  • Hero Member
  • *****
  • Posts: 973
Re: Updating OxyScheme
« Reply #8 on: November 05, 2019, 05:07:57 AM »
Hi Charles, Mike,

this is what I have found so far, I do not know if it really helps. I simply set a debugflag in o2scm.o2bas and followed the error of (1 + 1). The app crashes in function token at the first expression:     
sys c = skipspace()

I am not sure if the logic for skipspace() in o2scm.o2bas has changed between 0.1.0 and 0.2.8? Is there a function in the include files of version 0.2.8 which could be compared?

This works in version 0.1.0 and would be the expected output for version 0.2.8:

...
equal?
do
unless
when
#<EOF>
> (+ 1 1)
2
> (1 + 1)
Error: illegal function

>
in sub token Enter new expression
> (+ 1 1)

in sub token
2
> (+ 1 1)
2
>


Roland 
« Last Edit: November 06, 2019, 06:53:40 AM by Arnold »

Charles Pegge

  • Admin Support Member
  • *****
  • Posts: 4416
    • Oxygen Basic
Re: Updating OxyScheme
« Reply #9 on: November 05, 2019, 09:40:19 AM »
Aha! This line needs to be altered:

Code: [Select]
  #define clearinput()      currentline = endline = linebuff

assignment operator within an expression :=

Code: [Select]
  #define clearinput()      currentline = endline := linebuff


Arnold

  • Hero Member
  • *****
  • Posts: 973
Re: Updating OxyScheme
« Reply #10 on: November 05, 2019, 11:25:08 AM »
Thank you Charles. This was a very quick fix and the modification also worked for version 0.1.0. Although I studied Oxylog.txt several times I always overlooked your notes for 28/05/2019 (release 0.2.0). Maybe it is possible to create an error message for these cases? The rule should be clear, but careless mistakes will be unavoidable.

The adventure is not over yet. At the moment I investigate what happens after loading fib_fast.scm. (fibonacci 10) returns 1 which seems to be a bit too little. This will be a nice way to follow and study the code.

Roland

Charles Pegge

  • Admin Support Member
  • *****
  • Posts: 4416
    • Oxygen Basic
Re: Updating OxyScheme
« Reply #11 on: November 05, 2019, 03:25:21 PM »
The compound assignment vs comparation change was made after this discussion:

https://www.oxygenbasic.org/forum/index.php?topic=1936.0

#4..#15
« Last Edit: November 05, 2019, 03:34:21 PM by Charles Pegge »

Arnold

  • Hero Member
  • *****
  • Posts: 973
Re: Updating OxyScheme
« Reply #12 on: November 06, 2019, 06:57:18 AM »
Hi Charles, Mike,

I searched for some more compound assignments, perhaps I did not find all. Are my modifications correct, in particular line 555?

line 217:
#define clearinput()      currentline = endline := linebuff
line 555:
            args = envir := code := dump := NIL
line 1253:
    slow = fast := a
line 3344 - 3353
    car(NIL) = cdr(NIL) := NIL
    /* init T */
    flag(T) = (T_ATOM | DOMARK)
    car(T) = cdr(T) := T
    /* init F */
    flag(F) = (T_ATOM | DOMARK)
    car(F) = cdr(F) := F
    /* init EOF_OBJ */
    flag(EOF_OBJ) = (T_ATOM | DOMARK)
    car(EOF_OBJ) = cdr(EOF_OBJ) := EOF_OBJ

Attached is o2scm.o2bas as o2scm_mod.o2bas. At least I can now do something like:
(define y)

(write "Help") works, but (display "Help") does not work. What would be the best location in o2scm.o2bas for research?

Roland

Charles Pegge

  • Admin Support Member
  • *****
  • Posts: 4416
    • Oxygen Basic
Re: Updating OxyScheme
« Reply #13 on: November 06, 2019, 08:19:34 AM »
Thanks Roland.

There is a missing *

in printatom

Code: [Select]
    else if (isstring(l)) then
      if (flg == 0) then
        p = cast char strvalue(l)
      else

should be

Code: [Select]
    else if (isstring(l)) then
      if (flg == 0) then
        p = cast char* strvalue(l)
      else



Arnold

  • Hero Member
  • *****
  • Posts: 973
Re: Updating OxyScheme
« Reply #14 on: November 06, 2019, 02:47:17 PM »
Magnificent! This makes the difference between master and apprentice. I found printatom(), but I would never had found the cast char* solution. I will check the remaining cast cases too.

At the moment I try to find this difference:

> (if (= 10 0) (display "yes")
    (display "no"))
yes#t
>

The expected result is:
no#t
>

If I follow the code in o2scm.o2bas I end in about line 1535:
...
      case OP_IF1 /* if */
        if (istrue(value)) then
...

I am not sure if these two macros are still valid for newer versions of Oxygen:

  /* true or false value macro */
  #define istrue(p)  ((p) != NIL && (p) != F)
  #define isfalse(p) ((p) == NIL || (p) == F)

istrue and isfalse is used often in o2scm.o2bas but I do not yet know the expected result (1 or -1) and the values for nil, p and f. I will try to find this out tomorrow.