Save record to BLOB

[nanix]

Hello i wrote this code,but its got a bug that i cant find
i write to record but then if i save it OK if load it from BLOB OK.But if i close program and load it it won't load the things i saved.

Why is that? :wall:

Delphi-Quellcode:

TSensor=record
    Name:String[20];
    Value:String[20];
    Min:String[20];
    Max:String[20];
    Average:String[20];
    Delta:String[20];
    Offset:String[20];
  end;

  TMonitor=record
    Name:String[20];
    Sensor:Array [0..2] of TSensor
  end;

  TMonitoring=record
    Monitoring:Array [0..1] of TMonitor;
  end;



var
Monitoring:TMonitoring;
Monitor:TMonitor;
Sensor:TSensor;


procedure TDBSync.SavetoBLOB(ABSTable:TABSTable;Memo:TMemo);
var
   blobF : TBlobField;
   bs : TStream;
begin

 {for I := 0 to High(Monitoring.Monitoring) - 1 do begin
 for k := 0 to High(Monitor.Sensor) - 1 do begin     }

Monitoring.Monitoring[0].Name:='Monitor0';
Monitor.Sensor[0].Name:='Sensor0';
Monitor.Sensor[0].Value:='A';
Monitor.Sensor[0].Min:='B';
Monitor.Sensor[0].Max:='C';
Monitor.Sensor[0].Average:='D';
Monitor.Sensor[0].Delta:='E';
Monitor.Sensor[0].Offset:='F';

Monitoring.Monitoring[1].Name:='Monitor0';
Monitor.Sensor[1].Name:='Sensor0';
Monitor.Sensor[1].Value:='A';
Monitor.Sensor[1].Min:='B';
Monitor.Sensor[1].Max:='C';
Monitor.Sensor[1].Average:='D';
Monitor.Sensor[1].Delta:='E';
Monitor.Sensor[1].Offset:='F';

   ABSTable.Insert;

   blobF := ABSTable.FieldByName('BLOB') as TBlobField;
   bs := ABSTable.CreateBlobStream(blobF, bmWrite) ;
   try
     bs.Write(Monitoring,SizeOf(Monitoring)) ;

   finally

     bs.Free;
     ABSTable.Post;
   end;
end;



procedure TDBSync.LoadfromBLOB(ABSTable:TABSTable;Memo:TMemo);
var
   blobF : TBlobField;
   bs : TStream;
   i,k:integer;
begin
   if ABSTable.FieldByName('BLOB').IsBlob then
   begin
     blobF := ABSTable.FieldByName('BLOB') as TBlobField;
     bs := ABSTable.CreateBlobStream(blobF, bmRead);
     try
       bs.Read(Monitoring,sizeof(TMonitoring)) ;
     finally
       bs.Free;
     end;
   end;

 for I := 0 to High(Monitoring.Monitoring) - 1 do begin
 for k := 0 to High(Monitor.Sensor) - 1 do begin


 Memo.Lines.Add('* '+Monitoring.Monitoring[i].Name);
 Memo.Lines.Add(' -'+Monitor.Sensor[k].Name);
 Memo.Lines.Add(' -'+Monitor.Sensor[k].Value);
 Memo.Lines.Add(' -'+Monitor.Sensor[k].Min);
 Memo.Lines.Add(' -'+Monitor.Sensor[k].Max);
 Memo.Lines.Add(' -'+Monitor.Sensor[k].Average);
 Memo.Lines.Add(' -'+Monitor.Sensor[k].Delta);
 Memo.Lines.Add(' -'+Monitor.Sensor[k].Offset);
 end;
end;
end;

[Medium]

Line 60: I am not entirely sure about it, but this could be an issue. SizeOf(Monitoring) should be SizeOf(TMonitoring). I've got no Delphi at hand and cannot verify whether SizeOf() on a variable returns the size of it's type, or the size of the pointer to the structure. The latter would be what I belive can be the problem here.
You have it with the type in your loading procedure, but since the whole stream is only as short as the pointer, the data formerly stored in memory is not overwritten. This would be why you can access it when not closing the program in between - after restarting it, adresses usually change, and you didn't create the structure in the first place anyway then.

To make sure you actually write the data rather than a pointer, just check the stream's size after writing it. If it's only 4 bytes, you've only got the pointer.

It may just as well also be, that TMonitoring consists of cascading complex types itself. I think fixed-size arrays are actually written properly, but I am unsure if that is the case for the cascade of structures inside the array. Thus it may also be, that you just save the two 4 byte pointers inside the array that point to the rest. But I am not a 100% on whether Delphi does resolve records/arrays in records/arrays as it automatically does with the first-level record.

I defenitely need to do some more Delphi... I already start forgetting some details :(

[nanix]

Yes looks like only TMonitoring gets wrtitten to the BLOB and then to database.

Becouse when i save it and load while program ON i get this

BLOB Size is 924 Bytes constatntly

Zitat:

* Monitor0
-Sensor0
-A
-B
-C
-D
-E
-F
* Monitor0
-Sensor0
-A
-B
-C
-D
-E
-F

After resetting program i get this...

Zitat:

* Monitor0
-
-
-
-
-
-
-
* Monitor0
-
-
-
-
-
-
-

[Medium]

In that case, you'd have to write every sub-record/array yourself into the stream, and vice versa when loading. Another option would be to use a flat record that cointains all the fields directly, without structuring in sub-types. Of course this would lead to one looong record that won't be so conviniently usable with array indexes anymore.

I recommend writing a method that takes such a record as a parameter, manually writes every record/array in the desired order and returns that stream, ready to be saved. The same has to be done for loading of course. The biggest disadvantage of that is, that these methods have to be manually maintained when the record's structures change.

A more elegant version would involve using classes instead of records. Then you could implement something like a .ToStream() and .FromStream() method, which could use the RTTI to determin what to write/read, but taht would just be a bonus.

In Java there is another way of doing this: If a class implements the Serializable interface, it can easily be converted into a stream. If I recall correctly, there somewhere on these forums is a similar method for Delphi. That also might be a way for you.

In essence: Delphi does not resolve cascaded complex records/types in general by itself, you somehow have to write each yourself. There are multiple ways of doing this.

[nanix]

Now i can read and write BLOB but static arrays is trouble for me.I don't know how much stuff each computer has its not universal but then again i don't know how to save dynamic arrays to BLOB such as this..

How would i save this and load with dynamic array,i know i have to use setlength when saving.
But what to do when you close program and want to load :|

Delphi-Quellcode:

type
  TSensor = packed record
    Name    :String[20];
    Value   :String[20];
    Min     :String[20];
    Max     :String[20];
    Average :String[20];
    Delta   :String[20];
    Offset  :String[20];
  end;

  TMonitor = record
    Name   :String[20];
    Sensor :Array of TSensor
  end;

var
Monitor:TMonitor;
Sensor:TSensor;
FMonitoring : array of TMonitor;

[generic]

You are using a DB! Why are you not using normalized tables to store the data?

[nanix]

Speed is the first priority.I need DB Charts and aswell DB Reports.Other things i need DB for aswell..

With Normalized tables comes troubles with updating atleast in my case bcouse all data from component is generated from a loop.

[sx2008]

Zitat:

Zitat von nanix

Speed is the first priority.

Premature optimization is the root of all evil - Donald Knuth.
read more: http://www.codinghorror.com/blog/archives/000061.html
Maybe you save some time storing your data, but you lose very much time if you want to retrieve the data.
Assume you want to read the data for a specific sensor and you know the name of the sensor.
Doing it your way using BLOBs you have to read about 50% of all records to find your sensor.
This is called a full table scan.
A very bad approach.

"generic" is right; you have a database and you should not missuse the database.
Blobs are good for pictures, documents or even files but not for structured data.

And BTW, it seems that your table doesn't have a primary key.
Primary keys are really important.
Tables without primary keys and only a blob field performs very bad.
Without a primary key you can't even delete a record, because the database doesn't know which record should be deleted.

My advice is: throw away all your code and design a clean database with normalized tables.

[nanix]

I only save all data to 1 BLOB how can i loose performance with this. :lol:

[Medium]

Uhm, why then do you use a Database? :)