PC vs Mac for VDP processing

[p775]

I was just wondering if there was any data out there regarding the benefits or downfalls of processing with windows 7 vs tiger or snow leopard? Obviously you get more hardware for less money when going the PC route, but all of our licenses were with apple for our software, so our VDP processing machine became a g5 (circa 2007).

If there were major benefits of switching to a PC for the processing, I could probably have an indesign license purchased for a PC. Please post any experience or real life statistics/data if you have it. Thanks a lot!

[geoman]

What VDP software are you currently running?

[p775]

What VDP software are you currently running?

SORRY! I left out the most important details.... xmpie udirect standard with indesign cs4. I'm not tied to xmpie though...if it would help company efficiency, I'm sure I could gain access to an alternative.

[pcmodem]

I noticed you have said G5 Intel for the Mac you are using. It is either a G5 or a MacPro (Intel).

Since your Mac is 3 years old, it should be the MacPro and are either running Tiger or Leopard. I would upgrade to Snow Leopard to get 64-bit capability out of your OS. Also upgrading to InDesign CS5 for more 64-bit processing.

For VDP directly in the Native application (InDesign). I would stick to the platform the file originated on. If the customer designed it on the Mac, I would use a Mac for VDP. Like wise for the Windows files.

[p775]

I noticed you have said G5 Intel for the Mac you are using. It is either a G5 or a MacPro (Intel).

Since your Mac is 3 years old, it should be the MacPro and are either running Tiger or Leopard. I would upgrade to Snow Leopard to get 64-bit capability out of your OS. Also upgrading to InDesign CS5 for more 64-bit processing.

For VDP directly in the Native application (InDesign). I would stick to the platform the file originated on. If the customer designed it on the Mac, I would use a Mac for VDP. Like wise for the Windows files.

Hi, Thanks for the reply. The mac is intel based and is running tiger. We need something other than indesigns built in data merge because postal barcodes are a requirement of the projects that we run. All files are created on this machine itself, so we are not dealing with customer files.

[bky]

It's been years since I ran back-to-back tests using the same file, but I still have my performance notes from 2007.

The job was 1910 records, using Dynamic Print to optimized PDF, from a network server. All tests used Indesign CS3, XMPIE 3.5

Windows XP Pro, SP3,
Custom PC, Dual Athlon 1800s, RAID, 2Gb RAM.
Test 1: 18:17 (mm:ss)
Test 2: 18:48 (signs of memory leaking...restart)
Test 3: 16:04 (after restart)

Powerbook G4, 1.5Ghz, 2Gb RAM OSX 10.4.X
10:03 (other apps running)
9:46
10:01 (files copied locally)
8:30 (rebooted prior...memory leaking? local files)
8:59 (server files and data)
9:03 (another PBG4, rebooted before test)

Power Mac G5 dual processor, 2.0Ghz, 4Gb RAM, OSX 10.4.X
7:15 (multiple apps running)
7:09 (rebooted)

My conclusions were that XMPIE or InDesign (maybe both) were more efficient processing under OSX, even with far less hardware. This wasn't the result I was expecting: we all felt the 64-bit AMD CPUs would *kill* the G4 and even the G5. Based on the test results, I shelved plans to build an XMPIE cluster of cheap Windows machines. Most VDP jobs we do originate from Mac anyway, so this was an easy choice. On the Mac side, it was apparent that performance was directly tied to clock speed more than RAM, network, CPU bits, etc.

Later in 2007, I moved over main production to a quad-core Xeon Mac Pro 3.0Ghz. The same test in 2007 was finished in 2:34. (13Gb RAM, OSX 10.4.X, XMPIE 3.5, fresh reboot).
I also have a trick where I can run multiple instances of XMPIE to utlitlize all 400% of the CPUs, netting me 4X the performance for any given time. On straight addressing jobs with no graphics, I've seen a net speed of over 70,000 records per hour. Simple numbering has hit 250,000 rph.

I repeated this test in 2010 on my Mac Book Pro 3.06Ghz Core 2, under 10.5.X, and it finished in 2:28, indicating again that clock speed is more important that CPU type (Core2 vs Xeon was not relevant).

I have further testing to do with a new 8-way Mac Pro running 10.6.X. I'll have some data on that in a month or so, and I'll be comparing CS3, CS5, and using newer XMPIE plug ins. I locked into 10.4/CS3/XMPIE3.5 because newer versions of anything seemed to kill my multi-processor trick.

[p775]

It's been years since I ran back-to-back tests using the same file, but I still have my performance notes from 2007.

The job was 1910 records, using Dynamic Print to optimized PDF, from a network server. All tests used Indesign CS3, XMPIE 3.5

Windows XP Pro, SP3,
Custom PC, Dual Athlon 1800s, RAID, 2Gb RAM.
Test 1: 18:17 (mm:ss)
Test 2: 18:48 (signs of memory leaking...restart)
Test 3: 16:04 (after restart)

Powerbook G4, 1.5Ghz, 2Gb RAM OSX 10.4.X
10:03 (other apps running)
9:46
10:01 (files copied locally)
8:30 (rebooted prior...memory leaking? local files)
8:59 (server files and data)
9:03 (another PBG4, rebooted before test)

Power Mac G5 dual processor, 2.0Ghz, 4Gb RAM, OSX 10.4.X
7:15 (multiple apps running)
7:09 (rebooted)

My conclusions were that XMPIE or InDesign (maybe both) were more efficient processing under OSX, even with far less hardware. This wasn't the result I was expecting: we all felt the 64-bit AMD CPUs would *kill* the G4 and even the G5. Based on the test results, I shelved plans to build an XMPIE cluster of cheap Windows machines. Most VDP jobs we do originate from Mac anyway, so this was an easy choice. On the Mac side, it was apparent that performance was directly tied to clock speed more than RAM, network, CPU bits, etc.

Later in 2007, I moved over main production to a quad-core Xeon Mac Pro 3.0Ghz. The same test in 2007 was finished in 2:34. (13Gb RAM, OSX 10.4.X, XMPIE 3.5, fresh reboot).
I also have a trick where I can run multiple instances of XMPIE to utlitlize all 400% of the CPUs, netting me 4X the performance for any given time. On straight addressing jobs with no graphics, I've seen a net speed of over 70,000 records per hour. Simple numbering has hit 250,000 rph.

I repeated this test in 2010 on my Mac Book Pro 3.06Ghz Core 2, under 10.5.X, and it finished in 2:28, indicating again that clock speed is more important that CPU type (Core2 vs Xeon was not relevant).

I have further testing to do with a new 8-way Mac Pro running 10.6.X. I'll have some data on that in a month or so, and I'll be comparing CS3, CS5, and using newer XMPIE plug ins. I locked into 10.4/CS3/XMPIE3.5 because newer versions of anything seemed to kill my multi-processor trick.

Thanks a lot for the data! I guess we'll probably just be sticking with the g5 at this point. I've found that there will be no major gain in processing by upgrading the hardware; we would need to get xmpie's uimage in order to expedite the problem job. It's good to see the hard data though on the mac vs pc performance.

[bky]

If you are doing effects-heavy text, you have other options rather than uImage.
uImage's strong point is letter-by-letter, position-by-position image processing, which it does in Photoshop.
If you are doing simple text effects, you can actually do this in Applescript, driven from something like Filemaker Pro.

I did a project like this back in 2006, where we scripted the creation of some 2000 names into named JPEGs. They were put into an asset folder for HP Yours Truly, and we called the variable file name using a Quark template. Merge speeds were virtually instant, but the creation of the JPEGs took time (uImage will have the same issue..Photoshop will be the bottleneck). It was still pretty quick, and I could have clustered several machines to do the job. Even though I used JLYT for this project, it could easily be done in VIPP, PPML, VPS, etc. Using optimized PDF isn't a good fit for a workflow like that.

If you are interested, I can share some information on how to use 100% of each CPU for CS3/XMPIE3.5. On our 4-way Xeon, we can get almost 4X the performance of a single instance. It should work with your hardware/software. I've been able to scale almost linearly with processor cores.

I haven't tested Windows 7, nor newer CPUs, but I plan on doing that with the new 4-way Mac Pro we have (Hyperthreading shows 8 virtual cores).

[p775]

If you are doing effects-heavy text, you have other options rather than uImage.
uImage's strong point is letter-by-letter, position-by-position image processing, which it does in Photoshop.
If you are doing simple text effects, you can actually do this in Applescript, driven from something like Filemaker Pro.

I did a project like this back in 2006, where we scripted the creation of some 2000 names into named JPEGs. They were put into an asset folder for HP Yours Truly, and we called the variable file name using a Quark template. Merge speeds were virtually instant, but the creation of the JPEGs took time (uImage will have the same issue..Photoshop will be the bottleneck). It was still pretty quick, and I could have clustered several machines to do the job. Even though I used JLYT for this project, it could easily be done in VIPP, PPML, VPS, etc. Using optimized PDF isn't a good fit for a workflow like that.

If you are interested, I can share some information on how to use 100% of each CPU for CS3/XMPIE3.5. On our 4-way Xeon, we can get almost 4X the performance of a single instance. It should work with your hardware/software. I've been able to scale almost linearly with processor cores.

I haven't tested Windows 7, nor newer CPUs, but I plan on doing that with the new 4-way Mac Pro we have (Hyperthreading shows 8 virtual cores).

Hi, yes, I'd be very interested in your method! Again, this is a 2x 2.66 dual core machine. When I bring up the processor window, it shows all 4 cores in use while processing, but it sounds as though you may have some sort of optimization for this?

I'm not sure about the approach you mentioned, it sounds as though part/s were done manually? We would be pulling random first and last names from different spreadsheets every day for these, so the only salvation would be an alphabetical character set sliced from the background image I would think? Also, the text would need to expand/contract due to varying name sizes so that in itself would complicate matters further. We're currently exploring "3d" looking fonts and will be trying to come to a compromise with the client until we can get some sort of system in place. Just for reference, what now takes 20 minutes (3k records) would take roughly 3.3 HOURS - and that's just on the front processing end! (not including the docusp/igen side).

[tobias]

I seriously hate desktops for my vdp work. If you read into it enough you can prepare your vdp jobs in raw ppml and process those on a real server, keeping your desktop clear for facebooking or random lolcat-photo-surfing, and not requiring you to drink excessive amounts of coffee because you can't do anything else. The spec for PPML is free and open; everyone with a PC and a text editor (even notepad will suffice) can write ppml.

This does, however, require programming skill. The payoff is huge, though; both in flexibility and in speed. With a simple programming language such as PHP, you can create the outline for a VDP job in a few hours if you're new, and literally generate a job that's 100.000 impressions deep in under 30 seconds. Batched into chunks, if you like.

Combine it with the power of JDF and you're set to create huge jobs with the press of a button, on a daily basis.