|
SMPOV-Files / RenderAgent - Intern *
How things work internally
|
|
|
Whats this page about
/ Inhalt |
SMPOV and the RenderAgents communicate
with an simple cleartext-protocoll. If you know how it works,
you can make your own RenderAgents.
When developing the first version of SMPOV,
I thought of it like a bakery. SMPOV is the boss and the RenderAgents
are the employees. In our case the boss (SMPOV) gives "RenderJobs"
to the employees. This is done by writting out the "JOB-Files"
and additional informations in the COM-Folder.
Currently SMPOV does not tell a RenderAgent which Job to take
first or what to do. There are all the Jobs, each RenderAgent
can take any of these. SMPOV just waits till the Job is done.
Internally SMPOV waits till it finds a ".rdy"-File for
each RenderJob. Assume you have 625 tiles, then SMPOV waits for
these 625 ".rdy"-files before starting the RenderAgent.
On this page I'll desribe these things
more in detail.
Everything that is been exchanged between
any part of SMPOV is inside the so called "COM" (like
Communication) folder.
Actually inside the COM-Folder there are
three subfolders:
- BMP (for all resulting
BMP's
- JOB (contains the JOB-Files)
- INI (contains everything
needed by POV-Ray for rendering)
Additionally we can find a Folder for each
RenderAgent that has ever been accessing this COM-Folder. These
Folders are for private use of the renderAgents and currently
not used by SMPOV-Main. |
|
|
Step
1 / Schritt 1 |
When we drag one or more ".pov"-file(s)
into SMPOV, it is added to the SMPOV-Render-Que. SMPOV splits
the File in one or more "RenderJobs".
Inside this process the file gets a longer filename, to prevent
any confusion with files of the same name. This longer filename
contains the actual date as well as some random components.
- Example: The
file: "benchmark.pov" becomes internally
"benchmark_01082004_104626_87176...".
- All files from one ".pov"-Source-file
have the same starting-filename and diffrent endings.
We will refer to this internal
name further as FN.
|
|
|
Step 2 / Schritt
2 |
This is the JOB-Folder before any RenderAgent
is been activated. We see the following files:
- FN.info (not needed by RenderAgent,
contains the SMPOV-Settings for this file)
- FN.pnd.txt (not needed by RenderAgent,
this is for the PicPender.exe)
- FN_(tile-number)T.bat
(this is the later "Job-File",
RenderAgent is looking for!) there is one of these files for
each tile.
- FN_PP.bat (this file is not needed by
RenderAgent, it can be used to manually start the PicPender.exe).
Only one of these files, is used for RenderAgent.
We will take a closer look inside this
file later. On the left side, you can see that we have two JOB-Files
for this picture. Therefore we can see that is has to be rendered
in two tiles.
RenderAgent itself will reserve the file
by renaming it to a unique filename and as well as changing the
file-ending to ".job". Internally
there are few more steps to enshure that never
two RenderAgents can take the same RenderJob. The
key to this is that the file has been renamed to a unique Filename
and got another ending.
|
|
|
Step 3 / Schritt
3 |
At the same time, all Information that
are necessary for POV-Ray have been copied into the "INI-Folder".
These are:
- FN.pov as the POV-Ray source file to
render by all instances
- FN_(tile-number).ini as the
parameter-file to tell POV-Ray which part of the picture to
render and with which parameters. This file is later been changed
by RenderAgent, so we take a look at this later.
Thats all we need to start a distributed
Renderjob. |
|
|
Step 4 / Schritt
4 |
Before we start a RenderAgent.exe and
watch what will happen to these files. There are some important
aspects to think about:
- POV-Ray is installed possibly on another
location on the client where RenderAgent is running
- the COM folder has maybe another path
on the client then it has on the same machine
Therefore RenderAgent needs to do 2 steps
in the following:
- It needs to reserve a Job-File
for its own exclusive use (that is one of the FN_(tile-number)T.bat
files).
- RenderAgent will need to change
all pathes in the JOB and in the INI-File to the pathes
which are valid for this local client.
|
|
The Job File from SMPOV before it
has been changed by the RenderAgent |
cd "C:\Programme\POV-Ray
for Windows v3.5\"
C:
REM %POVEXE
/EXIT /RENDER "%MEPbenchmark_01082004_104626_87176_1_0x0-80x120a.ini"
REM REN "%MEPbenchmark_01082004_104626_87176_1_0x0-80x120a.ini"
*.rdy
REM Exit
bin\pvengine.exe /RENDER "f:\00_rpgm\01_bs\01_projekte\pb_smpov
4\com\ini\benchmark_01082004_104626_87176_1_0x0-80x120a.ini"
/EXIT
REN "f:\00_rpgm\01_bs\01_projekte\pb_smpov 4\com\ini\benchmark_01082004_104626_87176_1_0x0-80x120a.ini"
*.rdy
Exit |
The JOB-File is a batch file.
On the same machine it could have been started manually to start
the RenderJob (which was the case in first versions). On another
machine this would fail, cause the pathes would not be valid.
Therefore it contains some meta-information which are prefixed
by "REM" so they won't be used for SMP-Rendering. For
distributed rendering, these meta-informations will later be completed
by the RenderAgent with the actual pathes of the client. These
meta-informations are:
- %POVEXE. - which is the path and filename
of POV-Ray/Mega-POV on the client.
- %MEP which is the path to the COM\INI-Folder
|
|
The INI-File before it has been touched
by RenderAgent |
;
Diese Datei wurde automatisch erzeugt von SMPOV. 01-08-2004 / 10:46:28
; Diese INI-Datei ist Nr. 1 von insgesamt: 2
; Mode: POV-Ray 3.5 Mode
;
; Höhe des gesamten Bildes:
Height=120
;
; Breite des gesamten Bildes:
Width=160
;
; Start-Spalte des Bild-Ausschnittes:
Start_Column=0
;
; End-Spalte des Bild-Ausschnittes:
End_Column=80
;
; Input-Filename
; Path%F:\00_RPGM\01_BS\01_Projekte\PB_SMPOV 4\COM\INI\
Input_File_Name=F:\00_RPGM\01_BS\01_Projekte\PB_SMPOV
4\COM\INI\benchmark_01082004_104626_87176.POV
;
; Output-Filename
Output_File_Type=S
Output_File_Name=F:\00_RPGM\01_BS\01_Projekte\PB_SMPOV 4\COM\BMP\benchmark_01082004_104626_87176_1_0x0-80x120A.BMP
;
; Dieser Text wird aus der Common.ini übernommen
; und ist für alle Teilbilder gültig.
-D
Statistic_Console=off
|
In the INI-File there are
in the same way pathes for Input and Output-Filenname as well
as a necessary meta-information. The final INI will be produced
temporarily by RenderAgent out of these informations.
- Path% - this meta information shows
which part of the Input/Output-Filename has to be replaced by
the actual, valid path for the client.
Please note that this INI-File contains
all the settings given in the SMPOV-GUI and (appended) the common.ini
which is an optional feature in the SMPOV-GUI.. |
|
|
Step 4/ Schritt
4 |
Now we start a RenderAgent.exe and watch
what will happen to these files. |
|
What happened to the JOB-File? |
C:
start /wait bin\pvengine.exe /EXIT /RENDER "J:\INI\benchmark_01082004_104626_87176_1_0x0-80x120a.ini"
REN "J:\INI\benchmark_01082004_104626_87176_1_0x0-80x120a.ini"
*.rdy
DEL "J:\JOB\benchmark_01082004_104626_87176_1T.job"
EXIT
|
Taking a look into the RenderAgents
private Folder, we find a file called "_1_private.bat"
(Name is unimportant) which looks like we have expected. All pathes
are changed so the rendering will work on that particular client. |
|
|
Step 5/ Schritt
5 |
Taking a peek into the INI-Folder, we notice
that the INI-File for this RenderJob has been renamed and therefore
marked as "in use". |
|
|
Step 6/ Schritt
6 |
Thats all the RenderAgents have to do.
Let me put all the steps together, one after the other:
- Initialise:
- find the COM-Folder make a private
Folder inside
- find POV-Ray and Mega-POV, remember
their path
Then after the RenderAgents are running
what they do is this:
- Look for a JOB-File in COM\JOB.
- reserve it by moving it to the private
Folder.
- change the Job-File ...
- as well as the INI-File that belongs
to that JOB-File so the pathes fit to the client
- Look if the FN.pov is a MEGA-POV-File
or POV 3.5 (search for "#version ...")
- Run POV or MEGA-POV with the corrected
INI-File as paramter
- Wait till POV-Ray ends.
- Rename the "INI-File"-Ending
from ".ini" to ".rdy". This is the Signal
for SMPOV that the JOB is ready.
- RenderAgent will now open the CPU Slot
and continue with Step 1.
|
|
|
Additional comments |
The current Windows-Implementation of
RenderAgent (as well as SMPOV) is multithreaded, which makes the
source-code a biit more complex then necessary.
The above described operations overlap each other in real time.
Another implementation could be single-threaded without big disadvantages.
If someone has 4 CPU's he could start 4
of these single-threaded RenderAgents, while this is not necessary
for the current windows implementation. |
|
|
Additional Informations
about the SMPOV-Rendering Pipeline |
The following Informatios are not needed
to make a custom RenderAgent. They are provided for completeness. |
|
The benchmark_01082004_104626_87176.pnd.txt
|
SF:F:\00_RPGM\01_BS\01_Projekte\PB_SMPOV
4\COM\BMP\benchmark_01082004_104626_87176_1_0x0-80x120A.BMP
SF:F:\00_RPGM\01_BS\01_Projekte\PB_SMPOV 4\COM\BMP\benchmark_01082004_104626_87176_2_80x0-80x120A.BMP
DF:F:\00_RPGM\01_BS\01_Projekte\PB_SMPOV
4\COM\BMP\benchmark_01082004_104626_87176_2_80x0-80x120A.BMP
MV:F:\00_RPGM\01_BS\01_Projekte\PB_SMPOV 4\COM\BMP\benchmark_01082004_104626_87176_2_80x0-80x120A.BMP>F:\00_RPGM\01_BS\01_Projekte\PB_SMPOV
4\Ani-Test\benchmark.bmp
ER:F:\00_RPGM\01_BS\01_Projekte\PB_SMPOV 4\COM\JOB\benchmark_01082004_104626_87176.pnd.txt
|
The FN.pnd.txt contains
the instructions for the PicPender.exe how to combine the tiles
to one file. Even if you do not use tiling (so you have one single
picture result) the PicPender.exe is called to copy the resulting
file to its final destination. The
instructions work as follows:
- SF: - Sourcefile - each tile has one
Line SF: which marks the tile as a sourcefile. Please take a
look at the BMP-Filename which therefore contains additional
informations.
- DF: - Destinationfile - this can only
be given once and is typically a copy of the last Sourcefile.
- MV: - FileMove - this command makes
PicPender.exe to move a file from A -> B
- ER: - Erase File - given this command,
PicPender.exe will delete the the specidfied File, no matter
if its write-protected or not.
|
|
The benchmark_01082004_104626_87176_PP.bat
|
"F:\00_RPGM\01_BS\01_Projekte\PB_SMPOV
4\PicPend-1.exe" "F:\00_RPGM\01_BS\01_Projekte\PB_SMPOV
4\COM\JOB\benchmark_01082004_104626_87176.pnd.txt"
|
The file FN_PP.bat is the batchfile
that will be called from SMPOV after all Renderjobs have been completed.
It could be called manually as well, in case SMPOV would have been
ended before time. Currently it contains just one line, that is
the call to PicPender.exe with the FN.pnd.txt as parameter. |
|
The current "times.log" |
irid.pov
rendered in 3200x2400 in 16 Tiles (16,1)-(H) from 01-06-2004 / 23:51:44
to 01-06-2004/23:54:00. Tiling, copying ( 255 Sek.) ended: 01-06-2004
/ 23:58:15 (393 Sek.).
irid.pov rendered in 3200x2400 in 25 Tiles (1,25)-(V) from 01-06-2004
/ 23:58:52 to 01-07-2004/00:00:57. Tiling, copying ( 3 Sek.) ended:
01-07-2004 / 00:01:00 (131 Sek.).
irid.pov rendered in 9600x9600 in 10 Tiles (1,10)-(V) from 01-07-2004
/ 00:05:37 to 01-07-2004/00:12:03. Tiling, copying ( 68 Sek.) ended:
01-07-2004 / 00:13:11 (456 Sek.).
irid.pov rendered in 12288x12288 in 4 Tiles (1,4)-(V) from 01-07-2004
/ 00:23:19 to 01-07-2004/00:28:15. Tiling, copying ( 130 Sek.) ended:
01-07-2004 / 00:30:25 (427 Sek.).
irid.pov rendered in 3200x2400 in 2 Tiles (1,2)-(V) from 01-07-2004
/ 00:41:57 to 01-07-2004/00:42:21. Tiling, copying ( 3 Sek.) ended:
01-07-2004 / 00:42:23 (26 Sek.).
irid.pov rendered in 3200x2400 in 2 Tiles (2,1)-(H) from 01-07-2004
/ 00:42:34 to 01-07-2004/00:43:01. Tiling, copying ( 19 Sek.) ended:
01-07-2004 / 00:43:20 (46 Sek.).
benchmark1.pov rendered in 3200x2400 in 4 Tiles (1,4)-(V) from 01-07-2004
/ 01:07:00 to 01-07-2004/09:35:08. Tiling, copying ( 30 Sek.) ended:
01-07-2004 / 09:35:38 (30520 Sek.).
benchmark2.pov rendered in 3200x2400 in 4 Tiles (1,4)-(V) from 01-07-2004
/ 01:07:00 to 01-07-2004/09:35:43. Tiling, copying ( 17 Sek.) ended:
01-07-2004 / 09:35:59 (30540 Sek.).
Ani wood1.pov rendered in 160x120 in 1 Tiles (1,1)-(-) from 01-08-2004
/ 09:47:01 to 01-08-2004/09:58:57. Tiling, copying ( 1 Sek.) ended:
01-08-2004 / 09:58:57 (716 Sek.).
benchmark.pov rendered in 160x120 in 2 Tiles (2,1)-(H) from 01-08-2004
/ 10:46:28 to 01-08-2004/10:57:30. Tiling, copying ( 0 Sek.) ended:
01-08-2004 / 10:57:30 (662 Sek.). |
The "times.log"contains
informations how long each file has been in the SMPOV-Rendering-Pipeline,
as well as information how many seconds have been used for appending
the tiles and copying the final picture. There have been changes
about time-measurement before Distri 15, which make these results
more reliable, even if the rendering goes over several days or weeks. |
|