[ nedit-Bugs-1625283 ] Macro branch operations may crash on 64-bit systems

[SourceForge.net]

Bugs item #1625283, was opened at 2006-12-31 11:40
Message generated for change (Comment added) made by keidel
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=111005&aid=1625283&group_id=11005

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Program
Group: release
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Juergen Keidel (keidel)
Assigned to: Nobody/Anonymous (nobody)
Summary: Macro branch operations may crash on 64-bit systems

Initial Comment:
Building from Source on Linux with a 64-bit system leads to several error messages (casting from/to int from/to pointer.

After solving these compiler messages, the program runs. Using the preference file, it aborts with segfault when inserting something.
Without preferences (no Macros) it works.
Abort happens in ContinueMacro
at :  status = (*inst)();
The 32-bit version with same preference file runs stable.



----------------------------------------------------------------------

>Comment By: Juergen Keidel (keidel)
Date: 2007-01-03 11:15

Message:
Logged In: YES 
user_id=86514
Originator: YES


>From the file int.diff I see, that my Version of interpreter.c is old.
Is there any newer release available?


----------------------------------------------------------------------

Comment By: Juergen Keidel (keidel)
Date: 2007-01-03 11:09

Message:
Logged In: YES 
user_id=86514
Originator: YES

uname -a
Linux wizant2c 2.6.11.4-21.15-smp-pre #1 SMP Fri Dec 29 17:31:30 CET 2006
x86_64 x86_64 x86_64 GNU/Linux
gcc version 3.3.5 20050117 (prerelease) (SUSE Linux)

Hardware is intel core2duo Processor

First compile gave warnings about casting from/to pointer/int in follwing
file:
util/prefFile.c
highlight.c
interpret.h
parse.y
userCmds.c
interpret.c
macro.c
preferences.c

changing the int to long int let the warnings disappear.
Running nedit with smart indent resulted in segfault at insertion of first
character in a new line.
This happened in one of the Branch functions in interpreter.c  at a
statement like:
addr = PC + (int)*PC;
or:
PC += (long int)*PC;

replacing these by a sequence like:
    long int increm;

    increm = (((long int)*PC)<<32)>>32; /*   make a correct value */
    addr = PC +  increm;        

in branchTrue, branch, branchFalse, arrayIter
solved the problem.

In addition I had to change the makefile.linux, 
LIBS= -L/usr/X11R6/lib64 -Wl,-Bstatic -lXm -Wl,-Bdynamic -lXp -lXpm -lXext
-lXt -lSM -lICE -lX11 -lm

to link with the correct library.

No Its running on 64-Bit
The changed files are attached as  changed.tgz

File Added: changed.tgz

----------------------------------------------------------------------

Comment By: Scott Tringali (tringali)
Date: 2007-01-02 19:46

Message:
Logged In: YES 
user_id=11321
Originator: NO

The problem I see is that branching stuff in the parsing phase puts void*
onto the stack, but pulls them off as integers (see AddImmediate vs.
branchTrue).

I changed them both to integers and made it more type-safe with a union,
so the instructions won't take up any more space.  There seems to be no
reason for the cast to void* that I can find.  The file now compiles with
no warnings.

I think the downside is that you can only branch an int's worth, instead
of of large.  A better solution would to make all the branches be of
ptrdiff_t, but since macros are limited to a few K it doesn't matter.

Would so someone who knows this better take a look?



File Added: int.diff

----------------------------------------------------------------------

Comment By: Scott Tringali (tringali)
Date: 2007-01-02 18:37

Message:
Logged In: YES 
user_id=11321
Originator: NO

Need a bit more detail on your 64-bit system.  What CPU? Distro? Compiler?
Version of NEdit?

I don't dispute the code in there is wrong, it would help us to fix it if
we can actually repro it.  It doesn't repro for me on other 64-bit sytems I
have access to.

BTW: casting to long it will only put off the problem to another day for
another platform.

----------------------------------------------------------------------

Comment By: Juergen Keidel (keidel)
Date: 2007-01-02 10:06

Message:
Logged In: YES 
user_id=86514
Originator: YES

The reason for the segfault lies in interpreter.c, handling of branches.
getting a value from PC results in compilerwarnings about different sizes
of pointer and int. solving this by casting the void * of PC to long int
get an address-like expansion of the value (i.e. extended to
0x7fffffff00000008 p.E)
So the branching functions in interpreter.c must mask (or shift left32 and
right back 32-bit of the value)
This old trick of storing values in a pointer-location has problems at
64-Bit systems.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=111005&aid=1625283&group_id=11005