xorriso in ghostbsd-build

[ASX]

We will need to decomission HFS+ before the time stamps roll over
as 32 bit unsigned values by the added constant 2082844800 to the
seconds since 1970.

HFS+ epoch start 1 Jan 1904, and the timestamp allow for date-time up to 06 Feb 2040. The constant is the difference between hfs+ epoch and unix epoch.

By year 2040 HFS+ should be a thing of "historic interest" only, and that is a good reason to not decomission the related code, eventually adding a "warning" near hfsplus option in docs will let the user know about the (by then critical) limitation.

We have enogh time to discuss about, 22+ years

[scdbackup]

Hi,

HFS+ epoch start 1 Jan 1904, and the timestamp allow for date-time up to
06 Feb 2040.

Oh. Indeed.
FreeBSD will already have run into the signed integer seconds rollover
with ISO 9660:
https://github.com/freebsd/freebsd/blob ... 660_node.c

Code: Select all

  int
  cd9660_tstamp_conv7(pi,pu,ftype)
         u_char *pi;
         ...
  {
         ...
         y = pi[0] + 1900;

(In Linux for x86 there will be a years rollover in 2028 because its
equivalent of "*pi" is a char without explicitly given signedness.)

Have a nice day

Thomas

[ASX]

I have added a few printf() inside the code, to try to trace the process flow, and guess what ? at some point the error magically disappeared!
Removed some printf(), and the error reappeared.
Readded some printf(), and the error appear again. Very weird, because the printf() are supposed to be 'neutral'.
I have been unable to idenfify a specific problem so far, but I will recap what I was able to see until now:

the process flow (related to when the error happens):
...
xorriso/opts_d_h.c: Xorriso_option_end()
xorriso/opts_a_c.c: Xorriso_option_commit()
xorriso/write_run.c: Xorriso_write_session()

the output from a working session: (interrupted from me with CTRL C)

ASX write_session 18a *******************

ASX write_session 18e *******************

ASX write_session 18g *******************

ASX write_session 19 *******************

ASX write_session 20 *******************
libisofs: NOTE : Automatically adjusted MBR geometry to 1017/142/32

ASX write_session 21 *******************

ASX write_session 22 *******************

ASX write_session 23 *******************
xorriso : UPDATE : 0.59% done
xorriso : UPDATE : 7.97% done
xorriso : UPDATE : 16.40% done, estimate finish Fri Sep 22 11:43:17 2017
^C
UNIX-SIGNAL: SIGINT errno= 2

The output from a non working session:

ASX write_session 18 *******************

ASX write_session 18a *******************

ASX write_session 18e *******************

UNIX-SIGNAL: SIGSEGV errno= 22
xorriso : ABORT : Trying to shut down drive and library
xorriso : ABORT : Wait the normal burning time before any kill -9

xorriso : ABORT : Program done. Even if you do not see a shell prompt.

The code between printf() 18e and 18g:
xorriso/write_run.c

Code: Select all

printf("\nASX write_session 18e *******************\n");
   ret= isoburn_prepare_disc(source_drive, &disc, sopts);
 } else {
printf("\nASX write_session 18f *******************\n");
   ret= isoburn_prepare_new_image(source_drive, &disc, sopts, drive);
 }
printf("\nASX write_session 18g *******************\n");
 if(ret <= 0) {
   Xorriso_process_msg_queues(xorriso,0);
   sprintf(xorriso->info_text,"Failed to prepare session write run");
   Xorriso_msgs_submit(xorriso, 0, xorriso->info_text, 0, "FAILURE", 0);
   {ret= 0; goto ex;}
 }

printf("\nASX write_session 19 *******************\n");

apparently something is happening inside or aoround isoburn_prepare_new_image()

The behavior seen so far, in my experience usually derive from some unititalized variable/pointer.

to be continued ...

[scdbackup]

Hi,

apparently something is happening inside or aoround isoburn_prepare_new_image()

Yes. Despite their harmless names, the calls isoburn_prepare_disc() and
isoburn_prepare_new_image() start a thread which lets libisofs begin
to produce the ISO image stream.
This obviously fails early. Probably in the first pass which computes the
positions of the various entities inside the ISO.

The behavior seen so far, in my experience usually derive from some
unititalized variable/pointer.

valgrind would find such a thing if it was encountered in my tests.

We need the stack trace of the crash.


If you want to go on with inserting prints, then you should skip the
libisoburn layer and begin to mark code spots in libisofs.

The big boss in libisofs writing is
libisofs/ecma119.c : ecma119_image_new()
It runs in the thread of xorriso and near its end starts the libisofs
writer thread.
It controls the creation of the HFS+ writer objects:

Code: Select all

        ret = hfsplus_writer_create(target);
        ret = hfsplus_tail_writer_create(target);

These functions are implemented in
libisofs/hfsplus.c
You should put fprintf(stderr)s into them to see whether they are reached and
completely executed.

When all writers are created ecma119_image_new() runs the first pass by
calling their .compute_data_blocks() methods.
Those of HFS+ are in libisofs/hfsplus.c:

Code: Select all

hfsplus_writer_compute_data_blocks()

will print DEBUG messages if xorriso command
-report_about all
is given before
-as mkisofs
or after the "--" at the end of the arguments.
A problem might be that these messages go through a message queue and
might get lost unprinted when the process crashes.

Code: Select all

hfsplus_tail_writer_compute_data_blocks()

will print start and end messages if macro Libisofs_ts_debuG is defined.
These too go through the message queue.

So you will probably be best off with own fprintf(stderr, ...).

If we are still running after all .compute_data_blocks() methods were
called, the libisofs writer thread is started with write_function() as
executable payload.

It calls via write_head_part*() the HFS+ writer methods .write_vol_desc()
which are both the same and not very suspicious:

Code: Select all

nop_writer_write_vol_desc()

just returns 1.

Then it calls the .write_data() methods:

Code: Select all

hfsplus_writer_write_data()

writes the HFS+ superblock and tree. (That's were the aligment
warnings come from.)

After the successful end of hfsplus_writer_write_data() would come the
UPDATE messages about written percentage of data.
So i assume that its end is not successful.

Code: Select all

hfsplus_tail_writer_write_data()

writes HFS+ metadata after the data file content.

Finally via ecma119_image_free() it calls:

Code: Select all

hfsplus_writer_free_data()

cleans up memory allocations.

Code: Select all

nop_writer_free_data()

just returns 1.

Have a nice day

Thomas

[ASX]

scdbackup,

what I reported is nothing definitive ... the fact is I suspect that after adding debug info to be able to use a debugger, the issue might disappear ... that's why sometimes I prefer the printf(), I have more control over there. In the meantime I was able to go down a bit further and yes I'm arriving at ecma119_image_new() unfortunately from there I have difficult to replicate the problem, so far.

Thanks for the additional info, will try to make good use of them.

~~~

The more printf() I'm adding the more difficult is to replicate the problem ... it is going to take some time ... please be patient.
flow:
libisoburn/isoburn.c:int isoburn_prepare_new_image()
/libisoburn/isoburn.c: isoburn_prepare_disc_aux()
libisofs/ecma119.c: iso_image_create_burn_source()
ecma119_image_new()

going down further ...

[ASX]

posting for my own use ...

ASX **** ecma119_image_new 15g ************

ASX **** ecma119_image_new 15j ************

UNIX-SIGNAL: SIGSEGV errno= 22

Code: Select all

printf("\nASX **** ecma119_image_new 15j ************\n");
    /* create writer for HFS+/FAT structure */
    /* Impotant: It must be created directly before iso_file_src_writer_create.
    */
    if (opts->hfsplus || opts->fat) {
        ret = hfsplus_writer_create(target);
        if (ret < 0) {
//printf("\nASX **** ecma119_image_new 15k ************\n");
            goto target_cleanup;
        }
printf("\nASX **** ecma119_image_new 15l ************\n");
    }

uncommenting the line marked 15k lead to a successful execution ...

[scdbackup]

Hi,

uncommenting the line marked 15k lead to a successful execution ...

That's really strange. The printf() would only be reached if the
HFS+ writer creation had failed. In this case, no ISO image would
emerge. Not even the writer thread would be started.

I assume the message "15k" is not issued by the sucessful run.
Am i right ?

If so, then this looks like a plain compiler bug.

Does CLANG have options like -O2 ?
Would it help to set something like

Code: Select all

  export CFLAGS="-O0"

before running

Code: Select all

  ./configure && make

?

Have a nice day

Thomas

[ASX]

I assume the message "15k" is not issued by the sucessful run.
Am i right ?

Yes

If so, then this looks like a plain compiler bug.

the behavior I'm seeing is that I can add/remove some printf() and those are neutral, and sometimes I'm unable to replicate the issue ...

Does CLANG have options like -O2 ?

yes

Would it help to set something like

Code: Select all

  export CFLAGS="-O0"

before running

Code: Select all

  ./configure && make

will try ...

In the meantime I have arrived here:

ASX ** hfsplus_writer_create 1 ****

ASX ** hfsplus_writer_create 2 ****

ASX ** hfsplus_writer_create 3 ****

UNIX-SIGNAL: SIGSEGV errno= 22

code in libisofs/hfsplus.c :

Code: Select all

fprintf(stderr, "\nASX ** hfsplus_writer_create 2 ****\n");
    make_hfsplus_decompose_pages();
fprintf(stderr, "\nASX ** hfsplus_writer_create 3 ****\n");
    make_hfsplus_class_pages();
fprintf(stderr, "\nASX ** hfsplus_writer_create 4 ****\n");

to be continued ...

[ASX]

ASX *** make_hfsplus_class_pages 1 ***

ASX *** make_hfsplus_class_pages 2 ***

ASX *** make_hfsplus_class_pages 3 ***

ASX *** make_hfsplus_class_pages 3L ***

ASX *** make_hfsplus_class_pages 3L ***

ASX *** make_hfsplus_class_pages 3L ***

ASX *** make_hfsplus_class_pages 3L ***

ASX *** make_hfsplus_class_pages 3L ***

ASX *** make_hfsplus_class_pages 3L ***

ASX *** make_hfsplus_class_pages 3L ***

ASX *** make_hfsplus_class_pages 3L ***

ASX *** make_hfsplus_class_pages 3L ***

ASX *** make_hfsplus_class_pages 3L ***

ASX *** make_hfsplus_class_pages 3L ***

ASX *** make_hfsplus_class_pages 3L ***

ASX *** make_hfsplus_class_pages 3L ***

ASX *** make_hfsplus_class_pages 3L ***

ASX *** make_hfsplus_class_pages 3L ***

ASX *** make_hfsplus_class_pages 3L ***

ASX *** make_hfsplus_class_pages 3L ***

ASX *** make_hfsplus_class_pages 3L ***

ASX *** make_hfsplus_class_pages 3L ***

UNIX-SIGNAL: SIGSEGV errno= 22
xorriso : ABORT : Trying to shut down drive and library

code in libisofs/hfsplus_classes.c

Code: Select all

void make_hfsplus_class_pages()
{
    int page_idx = -1, char_idx, i;
    uint16_t *rpt, *page_pt;
    int page_count = 0;
fprintf(stderr, "\nASX *** make_hfsplus_class_pages 1 ***\n");
  
    memset(class_pages, 0, 19 * 256);
    for (i = 0; i < 256; i++)
        hfsplus_class_pages[i] = NULL;
fprintf(stderr, "\nASX *** make_hfsplus_class_pages 2 ***\n");
 
    rpt = (uint16_t *) class_page_data;
    page_pt = (uint16_t *) class_pages;
fprintf(stderr, "\nASX *** make_hfsplus_class_pages 3 ***\n");
    while (1) {
        if (*rpt <= page_idx)
    break;
        page_count++;
        page_idx = *(rpt++);
        char_idx = -1;
        while (1) {
            if(*rpt <= char_idx)
        break;
            char_idx = *(rpt++);
            page_pt[char_idx] = *(rpt++);
        }
        rpt++;
        hfsplus_class_pages[page_idx] = class_pages[page_count - 1];
        page_pt += 256;
fprintf(stderr, "\nASX *** make_hfsplus_class_pages 3L ***\n");
    }
fprintf(stderr, "\nASX *** make_hfsplus_class_pages 4 ***\n");
}

scdbackup, now I would need your help, about what to inspect next

[ASX]

export CFLAGS="-O0"

actually:

Code: Select all

env CFLAGS="-O0" ./configure

This setting make the problem disappear. So indeed, it seems an issue bound to CLANG optimizer.