Please help fixing mve_join script..

gustarballs1983

Vault Fossil
Modder
As the title says TeamX's application to join MVE clips has an issue that it only fuses togeher files that the final output is less than 3.5 or 4GB. if the final file would exceed this size the app only puts the second ingredient as the final video.

I belive it is due to the script that accompanies the app. could anyone look in the script and possibly tell me what d i need to change in order for te mve_join work with bigger files?

the interplay's MUTIL with MERGE option does fuse bigger files, however there's no way to reconvert them back to avi after that.
as MUTIL GETAVI complains about not supporting 16 bit audio yet, and MVE2AVI just crashes.

anyways, I'm posting mve_join's acompanying script:
Code:
{$apptype console}
uses windows, kol;

const BUF_SIZE = 100000;
var f1, f2, fo: file;
    buf: array [1..BUF_SIZE] of byte;
    to_read, bytes_read: integer;
    in_size: integer;

begin
    FileMode := 0;
    writeln ('MVE Joiner');
    if paramcount<>3 then begin
        writeln;
        writeln ('This program joins two Interplay MVE movies into one.');
        writeln;
        writeln (' usage: mve_join.exe movie1.mve movie2.mve movie_out.mve');
        exit;
    end;
    if not FileExists (paramstr(1)) then begin
        writeln ('Input file `', paramstr(1), ''' not found.');
        exit;
    end;
    if not FileExists (paramstr(2)) then begin
        writeln ('Input file `', paramstr(2), ''' not found.');
        exit;
    end;
    assign (f1, paramstr(1)); reset (f1, 1);
    assign (f2, paramstr(2)); reset (f2, 1);
    assign (fo, paramstr(3)); rewrite (fo, 1);

    writeln (' ', ExtractFileName(paramstr(1)), ' -> ', paramstr(3));
    in_size := system.filesize (f1);
    dec (in_size, 16); // cutting 16 bytes - EOF markers - from 1st file
    while (in_size > 0) do begin
        to_read := BUF_SIZE;
        if to_read > in_size then
            to_read := in_size;
        blockread (f1, buf, to_read, bytes_read);
        blockwrite (fo, buf, bytes_read);
        dec (in_size, bytes_read);
    end;

    writeln (' ', ExtractFileName(paramstr(2)), ' -> ', paramstr(3));
    seek (f2, $1A); // skipping MVE signature in 2nd file

    while not eof(f2) do begin
        blockread (f2, buf, BUF_SIZE, bytes_read);
        blockwrite (fo, buf, bytes_read);
    end;

    close (fo);
    close (f1);
    close (f2);
end.

any help @Lexx @NovaRain @phobos2077
??
 
That code doesn't have a size limit inside of it so you're hitting some sort of OS limit.
You mention 4GB. I am wondering if you're doing this on a FAT32 disk because maximum file size on that file system is exactly 4GB.

P.S. It's a long shot, but perhaps this could help if you can run the exe in Windows as a native application.
https://www.techpowerup.com/forums/threads/large-address-aware.112556/

No man, one couldn't install WinOS 64bit on a FAT32 filesytem, as there are files larger than 32 bits already in the system.

LAA is a dud shot yet agin hexer, don't mistake RAM / Virtual memory mapping, with Disk filesize limitation. as i said Interplay's MUTIL *is* able to handle output files larger than 4GB despite beeing a 16 bit application, so mve-join not working with large files is not an physical issue but rather issue with how it has been programmed.

a.k.a the file is beeing processed frame by frame, and written to disk frame by frame, so no need to have LAA, as it will never require more than few MB to handle a single frame of 640x480... on the other hand NTFS which I'm on handles large files up to several TB's quite nice.
 
So I joined just to reply to this, specifically. Long time lurker....

Not sure if you've figured out you're issue or not. Looking through your code, it looks like the bytes_read might be overflowing when it reaches the high side of an unsigned 32-bit integer e.g. a basic integer (even on a 64-bit system, asking for an int/integer is still usually fulfilled at runtime by a uint32 unless specifically asking for a uint64 something about memory and optimization :eyeroll:). An unsigned integer has a byte range of 0 to 4,294,967,295 -- or as you stated above cutting your files off at 4GB -- 4,294,967,295 bytes.

I'm not familiar enough with this scripting tool, but if it's possible, you'll probably need to change bytes_read to an unsigned 64-bit integer.

Hope this helps.

- Deuce

Edit: source code pseudo formatting

Edit 2 (6/5): so upon investigation of the actual code from the database, I discovered the “companion script” is actually the source code. From what I gathered it is a Delphi Pascal project file circa 2003-ish based on the date of the exe build date. I converted it to .pas and tried compiling with Open Pascal and things were wonky. Specifically, the buffer would only max out to a 16-bit integer — 32767, significantly less than 100K bytes. I was, however, able to confirm that an unsigned integer in pascal ranges 0-4,294,967,295. Updating the code, the object type for bytes_read should be changed to int64 and the application can be recompiled. When I did this and tried joining regular MVE files, everything worked.

Code:
{$apptype console}
uses windows, kol;

const BUF_SIZE = 32767;

var f1, f2, fo: file;
    buf: array [1..BUF_SIZE] of byte;
    to_read: integer;
    bytes_read: int64;
    in_size: integer;

begin
    FileMode := 0;
    writeln ('MVE Joiner');
    if paramcount<>3 then begin
        writeln;
        writeln ('This program joins two Interplay MVE movies into one.');
        writeln;
        writeln (' usage: mve_join.exe movie1.mve movie2.mve movie_out.mve');
        exit;
    end;
    if not FileExists (paramstr(1)) then begin
        writeln ('Input file `', paramstr(1), ''' not found.');
        exit;
    end;
    if not FileExists (paramstr(2)) then begin
        writeln ('Input file `', paramstr(2), ''' not found.');
        exit;
    end;
    assign (f1, paramstr(1)); reset (f1, 1);
    assign (f2, paramstr(2)); reset (f2, 1);
    assign (fo, paramstr(3)); rewrite (fo, 1);

    writeln (' ', ExtractFileName(paramstr(1)), ' -> ', paramstr(3));
    in_size := system.filesize (f1);
    dec (in_size, 16); // cutting 16 bytes - EOF markers - from 1st file
    while (in_size > 0) do begin
        to_read := BUF_SIZE;
        if to_read > in_size then
            to_read := in_size;
        blockread (f1, buf, to_read, bytes_read);
        blockwrite (fo, buf, bytes_read);
        dec (in_size, bytes_read);
    end;

    writeln (' ', ExtractFileName(paramstr(2)), ' -> ', paramstr(3));
    seek (f2, $1A); // skipping MVE signature in 2nd file

    while not eof(f2) do begin
        blockread (f2, buf, BUF_SIZE, bytes_read);
        blockwrite (fo, buf, bytes_read);
    end;

    close (fo);
    close (f1);
    close (f2);
end.
 
Last edited:
nah this code is inside an .ini file so i guess it does not require compilation, so i just need to change "bytes_read" into "int64" ?

anyways i'll try to keep in mind your solution, and edit the .ini accordingly nest time i'll have to deal with such large files.
Fortuneatly i got my hands on a newer version of the program i was running, and it allows me to specify the output bitrate now. As previously the program chose a bitrate of it's own, producing outrageusly large videos.

EDIT:
Yeah I forgot to say this.. but a Grand ThankYou to to You TheDeuceII


EDIT#2 it doesn't work

I've replaced the iteger with int64 and the same thing happens only the second video gets taken into creation of the joint video.

although i've found a second thing that may be the problem here:

Code:
        to_read := BUF_SIZE;
        if to_read > in_size then
            to_read := in_size;

this little code forces it to use 32 bit value
 
Last edited:
nah this code is inside an .ini file so i guess it does not require compilation, so i just need to change "bytes_read" into "int64" ?

anyways i'll try to keep in mind your solution, and edit the .ini accordingly nest time i'll have to deal with such large files.

...

Hey Guster,

What ini file are you speaking of? Is there another exe out there I'm unawares of? I'd like to find it and help fis it if I can!

I couldn't find anything but mve_joiner_v0.1_byABel.zip on the database, and that contains that Delphi Pascal project file I mentioned in my edit (.dpr). Anyway, I compiled the new version if you want to give it a shot for me and see if it works or not... Unsure if a new user can upload here or not, but here's a link:

EDIT: LINK REMOVED BY AUTHOR
 
Last edited:
MVE_JOIN_error.jpg


dunno what's up..

I got this error..

w8 wasn't it You who made that shitfest in Yesterday's thread few days before rus-ukra war started, after warlockracy mocked of Fo:Yesterday's project? I certainly remember that guy was called theDeuce..

and yeah i ment the .dpr file.
i opened it with notepad++ and saw the code, so i thought it was some kind of configuration file not the source code..
anyways why is your file over 400k in size while the original is only 19k?
 
Last edited:
View attachment 28723

dunno what's up..

I got this error..

w8 wasn't it You who made that shitfest in Yesterday's thread few days before rus-ukra war started, after warlockracy mocked of Fo:Yesterday's project? I certainly remember that guy was called theDeuce..

and yeah i ment the .dpr file.
i opened it with notepad++ and saw the code, so i thought it was some kind of configuration file not the source code..
anyways why is your file over 400k in size while the original is only 19k?

Nope, 'twasn't me, mate... I don't comment on Youtube, sorry.

But anyway the error...

Hmm... Is there any way you can share some large files to me? All I have is very small MVE from FO1/2 folders (e.g. BOIL1.MVE, BOIL2.MVE, BOIL3.MVE, etc...) and from Fallout Et Tu.

As for the 400k filesize... That's probably a compiler config/optimization thing... I'm not a Pascal developer. I live in the C#, Python, Java, world. I pulled down the OpenPascal compiler, made the changes based on the documentation from the internets, and then compiled it for Windows x86_64.

I'm really thinking of rebuilding this whole thing in Python and open sourcing it on Github.

EDIT #1: I changed the compiler config and got the size down to 72K and also got it working on my machine... I was able to reproduce the error. Seems to have been a compilation error (??? I really don't know for sure ???). Again, not a Pascal dev, so I don't know if the '.o' file is needed, but I included it as well. It might be an OpenPascal thing (?). Anyway, here is a link if you want to try it out again:

EDIT: LINK REMOVED BY AUTHOR

EDIT #2: wow I can see why you remember that guy. Fuck him. It was PeaceDuke by the way, or Puke for short. I can assure you and promise you it WAS NOT me. Starts here-ish, though: https://www.nma-fallout.com/threads/fallout-yesterday.219083/page-26#post-4462892
 
Last edited:
the vid is gone.. and i was talking rather about a quarrel about the video here on NMA in Fo:Yesterday's thread..

anyways, the program works the same way as the origial one takes the second video only.. and i think i know why let's look at the source code now:

Code:
{$apptype console}
uses windows, kol;

const BUF_SIZE = 32767;

var f1, f2, fo: file;
    buf: array [1..BUF_SIZE] of byte;
    to_read: integer;
    bytes_read: int64; <---Bytes Read 64bit integer
    in_size: integer; <--- in size 32bit integer

begin
    FileMode := 0;
    writeln ('MVE Joiner');
    if paramcount<>3 then begin
        writeln;
        writeln ('This program joins two Interplay MVE movies into one.');
        writeln;
        writeln (' usage: mve_join.exe movie1.mve movie2.mve movie_out.mve');
        exit;
    end;
    if not FileExists (paramstr(1)) then begin
        writeln ('Input file `', paramstr(1), ''' not found.');
        exit;
    end;
    if not FileExists (paramstr(2)) then begin
        writeln ('Input file `', paramstr(2), ''' not found.');
        exit;
    end;
    assign (f1, paramstr(1)); reset (f1, 1);
    assign (f2, paramstr(2)); reset (f2, 1);
    assign (fo, paramstr(3)); rewrite (fo, 1);

    writeln (' ', ExtractFileName(paramstr(1)), ' -> ', paramstr(3));
    in_size := system.filesize (f1);
    dec (in_size, 16); // cutting 16 bytes - EOF markers - from 1st file
    while (in_size > 0) do begin
        to_read := BUF_SIZE;
        if to_read > in_size then ; If "to read" is the whole first file which is over 2GB is more than  32 bit integer, than
            to_read := in_size; to read equals 32bit integer ( hence why it only takes the smaller part
        blockread (f1, buf, to_read, bytes_read);
        blockwrite (fo, buf, bytes_read);
        dec (in_size, bytes_read);
    end;

    writeln (' ', ExtractFileName(paramstr(2)), ' -> ', paramstr(3));
    seek (f2, $1A); // skipping MVE signature in 2nd file

    while not eof(f2) do begin
        blockread (f2, buf, BUF_SIZE, bytes_read);
        blockwrite (fo, buf, bytes_read);
    end;

    close (fo);
    close (f1);
    close (f2);
end.

this needs to change too i think and i think buffer size must be bigger as well, what i do know is that interplay tool does it correctly and joins bigger files no matter the size, so if you're going to tackle this program it would be cool if you'd tackle the others too mve2avi16.exe and mve2exe.exe as well as neither of theese work when the file is over 4GB and merged by interplay's MUTIL.exe with MERGE function

EDIT I get it!!!
the "to read" is both input files files so something over 2.8GB plus something over 900MB that's seposedly more than an "integer" and that's why it only take sthe second part, because the whole file would be greater than an integer,, this needs to change..
i propose:

Code:
{$apptype console}
uses windows, kol;

const BUF_SIZE = 100000;

var f1, f2, fo: file;
    buf: array [1..BUF_SIZE] of byte;
    to_read: int64;
    bytes_read: int64;
    in_size: int64;

begin
    FileMode := 0;
    writeln ('MVE Joiner');
    if paramcount<>3 then begin
        writeln;
        writeln ('This program joins two Interplay MVE movies into one.');
        writeln;
        writeln (' usage: mve_join.exe movie1.mve movie2.mve movie_out.mve');
        exit;
    end;
    if not FileExists (paramstr(1)) then begin
        writeln ('Input file `', paramstr(1), ''' not found.');
        exit;
    end;
    if not FileExists (paramstr(2)) then begin
        writeln ('Input file `', paramstr(2), ''' not found.');
        exit;
    end;
    assign (f1, paramstr(1)); reset (f1, 1);
    assign (f2, paramstr(2)); reset (f2, 1);
    assign (fo, paramstr(3)); rewrite (fo, 1);

    writeln (' ', ExtractFileName(paramstr(1)), ' -> ', paramstr(3));
    in_size := system.filesize (f1);
    dec (in_size, 16); // cutting 16 bytes - EOF markers - from 1st file
    while (in_size > 0) do begin
        to_read := BUF_SIZE;
        if to_read > in_size then
            to_read := in_size;
        blockread (f1, buf, to_read, bytes_read);
        blockwrite (fo, buf, bytes_read);
        dec (in_size, bytes_read);
    end;

    writeln (' ', ExtractFileName(paramstr(2)), ' -> ', paramstr(3));
    seek (f2, $1A); // skipping MVE signature in 2nd file

    while not eof(f2) do begin
        blockread (f2, buf, BUF_SIZE, bytes_read);
        blockwrite (fo, buf, bytes_read);
    end;

    close (fo);
    close (f1);
    close (f2);
end.

this way there shouldn't be any problems even if one of the inputs is over 4GB not only output. additionally the buffer size is back to waht it used to be, so that buffer is not overfilled ( possibly it will do it's work faster. i initially thought that only by changing the .dpr file one will succed in fixing the file, however i guess You're right and it needs to be compiled first.. so will Ya?

edit as for the files to test there are plenty of big .mve files on my ftp server Look in FoYesterday's folder..
the ftp server seemes to be down.. i'll just "kick the router in a special way" and it should fix the issue..
 
Last edited:
Yeah sorry for the delay.. works like a charm.

The problem is videos joined with the v0.4 can't be converted to standalone player with mve2exe nor can they be converted back to avi with fsp (a.k.a mve2avi16).
it just spits an error and doesn't do nothing.

Care to decompile theese and take a look?

Even ffmpeg has problems converting those videos back to .avi. ffmpeg does convert to avi yet messes up the video parameters. fps, bitrate, and lots of other stuff are taken straight out of ass by ffmpeg, guess i'd have to specify all this stuff in order to make a proper .avi yet ffmpeg does not display mve's bitrate anywhere, it just states 0.0000 and for the avi it assumes 200kb. the fps can be read from ffmpeg though. it states them correctly not sure if one can trust ffmpeg judgment regarding pixel and color format. and yet the time to convert is literally ages..
 
Yeah sorry for the delay.. works like a charm.

The problem is videos joined with the v0.4 can't be converted to standalone player with mve2exe nor can they be converted back to avi with fsp (a.k.a mve2avi16).
it just spits an error and doesn't do nothing.

Care to decompile theese and take a look?

Even ffmpeg has problems converting those videos back to .avi. ffmpeg does convert to avi yet messes up the video parameters. fps, bitrate, and lots of other stuff are taken straight out of ass by ffmpeg, guess i'd have to specify all this stuff in order to make a proper .avi yet ffmpeg does not display mve's bitrate anywhere, it just states 0.0000 and for the avi it assumes 200kb. the fps can be read from ffmpeg though. it states them correctly not sure if one can trust ffmpeg judgment regarding pixel and color format. and yet the time to convert is literally ages..

Looks like ABel wrote that MVE2AVI16 app, too. My guess is that was written in pascal too, hopefully. Probably size limited on the in and out file sizes is my guess. I don’t know. I’ll see about pulling the files down and see what’s there and see about decompiling them tonight!

EDIT #1: So I installed ffmpeg, and encoded my 12GB file with it. Granted, it is only BOIL1.MVE from Fallout 1 repeated 26 times, but the filesize is what matters. It took five minutes to encode to AVI for me... I'm not sure if this is a processor or perhaps an old version issue but the command I ran was
Code:
ffmpeg.exe -i BOIL12.MVE output.avi
and came out to just a bit over 1GB (~1/12th the size). Mby it's the shit quality video to start with and the nothing burger encoding I'm using... I don't know what you're trying to do exactly. I'm sorta have an idea based on what a scraped together on FO:Y forum, but I don't know your specs :)

 
Last edited:
ok.. perhaps it's the whole night spent in front of the PC, but i haven't understood a single sentence you wrote in the last post...

forget ffmpeg.. it does a sloppy job of converting to avi, lots of stuff is fucked in the avi wen it's made by ffmpeg.. better focus on mve2avi.
ffmpeg will not retain the correct bitrate, it will not retain the correct frps on the output avi, leading to the video beeing fucked.

avi2mve did the job good for smaller videos however for the ones merged with the new v0.4 of mve_join and the ones merged with interplay's MUTIL.EXE MERGE there is no good option to convert back to avi. MUTIL.EXE GETAVI complains it doesn't operate on 16 bit audio, ad exits out,and mve2avi just crashes, on the other hand ffmpeg spits out a broken turd, so for the time being there are no option to convert back larger files
 
An update:

I've pulled down all, and I mean all of the MVE-related tools that are in the database (Fallout Database | No Mutants Allowed (nma-fallout.com)). So far, I haven't had much luck decompiling the MVE->AVI or AVI-MVE converters, however I have started porting the MVE-Joiner to python as I mentioned I was thinking about doing...

I think the other converters can be done, too. Maybe in Python, maybe in C#... Either way, they're gonna be open source so the community can work on them and make them better. More on that later.

I found the byte conversion for MVE files, and that should help decompiling the video files in code into AVI files. Then we just need to reverse that... Not an easy task, but I'm sure there's other devs around here that know more than me that might want to help :P

Interplay MVE file format (github.com)
 
that's "ubercool"

Thanks Guster...!

Making some progress with the Joiner right now. Have a repository stood up and some code started. I haven't had a lot of free time to do much, but I had a few hours tonight, so I thought I'd work a bit on it. We can split the Interplay header out of the MVE, now. So that's progress (see the file format link I posed above)

I hope to get something for early alpha testing, soon. At least for the Joiner. My goal is to make it 1:1 recreated and perhaps add the ability to batch join... If that'd be helpful :shrug:

Anyhow, I'll check in soonish and put out a call to action for help coding up the MVE->AV, AVI->MVE, and MVE->EXE ports... The biggest thing we need is someone familiar with how the MVE file format works
 
The part about "someone versed in .mve format..
Bruh i think your best bet is tim cain himself.
However i highly doubt he'll want to be a part of reverse engineering copyprotected stuff of the company he used to be a part of
 
Back
Top