Code Optimization

[sdean]

Hi i've these functions to Store Streams in One File :

Delphi-Quellcode:

function AddData(ID:string; pBuf: Pointer; Count: Integer): Integer;
var
  Buff   : Pointer;
begin
 // FFile is of Type : TFileStream
  FFile.Seek(0, soFromEnd);//--> the Container * 
  if( Result > -1 )then
  begin
   GetMem(Buff, Count);
   try
    Move(pBuf^, Buff^, Count);
    Result := FFile.Write(Buff^, Count);
   finally
    FreeMem(Buff, Count);
   end;
  end;
end;
{* and here how i add Streams *}
function InsertStream(ID: String;aStream: TStream): Integer;
var
  pBuff : Pointer;
begin
  // alloc Mem for pBuff
  GetMem(pBuff, aStream.Size);
  try
   aStream.Position := 0;
   aStream.Read(pBuff^, aStream.Size);
   // now add data
   Result := AddData(ID, pBuff, aStream.Size);
  finally
   FreeMem(pBuff, aStream.Size);
  end;
end;

my question is :
how could i optimize the InsertStream Function so that it can work fast for file's size more than 1 GB cause i cannot load such a huge size in Memory.


many thanks in advance

[FredlFesl]

The Code contains errors and redundant stuff.
Here's my improved / modified and untested version.

Delphi-Quellcode:

Const
  BLOCKSIZE = 1024*1024; // Write in chunks @ 1MB

function AddData(ID:string; pBuf: Pointer; Count: Integer): Integer;
begin
  FFile.Seek(0, soFromEnd);
  if( Result > -1 )then // ***** WHERE DOES THIS Result COME FROM? if you leave it Like this, it will not work.
    Result := FFile.Write(PAnsiChar(pBuf)^, Count);
end;

function InsertStream(ID: String;aStream: TStream): Integer;
var
  pBuff : Pointer;
  BytesRead : Integer;

begin
  GetMem(pBuff, BLOCKSIZE); // allocate a chunk of memory
  try
   aStream.Position := 0;
   Repeat
     BytesRead := aStream.Read(pBuff^, MIN (BLOCKSIZE, aStream.Size - aPosition)); // move data into the chunk
     Result := AddData(ID, pBuff, BytesRead); // Add the chunk to the file
   Until (Result<>0) or (BytesRead < BLOCKSIZE);
  finally
   FreeMem(pBuff, BLOCKSIZE);
  end;
end;

However, a much simpler approach would be

Delphi-Quellcode:

FFile.Seek(0, soFromEnd);
FFile.CopyFrom (aStream, FFile.Position);

[sdean]

thank you FredlFesl

Delphi-Quellcode:

BytesRead := aStream.Read(pBuff^, MIN (BLOCKSIZE, aStream.Size - aPosition));

aPosition >> do you mean aStream.Position

[Luckie]

Why is FFile global? If it is global because it is needed in different parts of the code, why don't you make a class out of the code?

[sdean]

thank you Luckie :
any help of that !

[FredlFesl]

Zitat:

Zitat von sdean (Beitrag 1099856)

thank you FredlFesl

Delphi-Quellcode:

BytesRead := aStream.Read(pBuff^, MIN (BLOCKSIZE, aStream.Size - aPosition));

aPosition >> do you mean aStream.Position

Yes.

[Luckie]

The class could look like this:

Delphi-Quellcode:

TXyz = class
private
  FFile: TFileStream;
public
  property XyzFile: TFileStream read FFile write FFile;
  procedure AddData(...);
  procedure InsertData(...);
end;

[sdean]

but FredlFesl : your improved / modified and untested version writes Only 1MB to FFile
i mean the FFile will only hold 1MB of the Source Stream not the Whole Stream size .

[sdean]

Zitat:

Zitat von Luckie (Beitrag 1099864)

The class could look like this:

Delphi-Quellcode:

TXyz = class
private
  FFile: TFileStream;
public
  property XyzFile: TFileStream read FFile write FFile;
  procedure AddData(...);
  procedure InsertData(...);
end;

Thank you Luckie , Could you please explain :
what s the Diff between Declaring FFile as global and Creating this New Class as long as the Result will remain the same : the Write / Read Operations will be focusing on the FFile .

and many thanks .

[Luckie]

The result will of course be the same. But encapsulating the code makes it more readable, easier to administrate and (re)use.

[rollstuhlfahrer]

Zitat:

Zitat von sdean (Beitrag 1099865)

but FredlFesl : your improved / modified and untested version writes Only 1MB to FFile
i mean the FFile will only hold 1MB of the Source Stream not the Whole Stream size .

This code actually copies the whole Stream, not just 1MB. The 1MB is the chuncksize. In this code there is a Repeat-Until-Structure, which repeats until the whole stream is copied.

Bernhard

[FredlFesl]

Zitat:

Zitat von sdean (Beitrag 1099865)

but FredlFesl : your improved / modified and untested version writes Only 1MB to FFile
i mean the FFile will only hold 1MB of the Source Stream not the Whole Stream size .

No. It writes the whole file. Notice the "REPEAT UNTIL" loop. It reads 1MB data from the source and appends it to the file. Then in reads the next 1MB.... until there is no more data in the source.