hmm... what do you think about this?

http://www.spectracal.com/Documents/CM_LS.pdf

andy

It is the biggest load of bull-shit we have seen for some time...
We have pointed errors out to Spectracal, and Scott, the author, but they have basically ignored us.
Without providing the data used for the testing the results are at best inaccurate, and at worse just marketing hype.
And the reality is it is NOT comparing Calman, as it is impossible to import 'data sets' into Calman, so there is no way to have tested it in the way described!

Total bollox!

See:

http://www.lightillusion.com/lightspace_luts.html

http://www.lightillusion.com/guesswork.html

http://www.lightillusion.com/trade-ups.html

Steve

You can signup (http://www.spectracal.com/forum) or login to Spectracal's Forum to take a look for how many reported problems users have posted.

This shows just how many problems Calman users have with calibration, and that such problems have been unsolved for years.

If you read the Light Illusion forum posts (or other calibration forums) you will not see any such on-going problems with LightSpace.

And see also the first feedback from users that runned the latest Calman with that new revolutionary technology that had better results with older Calman versions.

This paper has a lot of errors, I have reported a first wave of questions on another forum...

Andy, I'd be very suspicious of the report's accuracy. I wasted hundreds of hours during the summer of 2012 working with Spectracal tech and the then new Studio software. It never worked properly. Two things bothered me. One, there was a go between rep from me to the tech department. I would report the errors but they NEVER got back to me. The go between would lead me to a new version, that wouldn't work either, report, again no feedback from tech. Two, all the while this was going on their publicity department was busy proclaiming how good the software was and how many studios were now using it, etc. This was pure BS.

While it's true that the present version of Calman can produce LUTs without the failures of the past, I don't believe it has a chance of being more accurate than LightSpace.

I use ChromaPure Pro and Calman Ultimate for some calibration duties, but for LUTs I use LightSpace full CMS.

Buzz

Andi,

in addition ... I ask myself over and over again ... why was that "scientific looking" comparison report just be done using a low end probe instead of e.g. a K10-A or something else fast + reliable probe, to get every kind of fluctuation influences (probe, display) lowered as much as possible?? ... not really professional for making such a deep-detailed comparison report, who can result into far-reaching implications because of possible misstatements!

Clear - SC's intention is just to get LS look poor compared with their own product. IMHO that report is just another kind of propaganda battle ...
Not really nice.

thanks for the replys!

that's what i suspected...

I've just been looking through the Spectracal forums, and there are reports of major problems with the CM build this report claims to have used...

Either the report is fake, or the results are confirmed as crap as they couldn't pick up the major issues users are finding!

Just to finish this thread, I posted the following on another forum, where Spectracal have been trying to use the document as marketing hype.
This should explain the reality.



I think the major issue with the 'document' (I can't call it a 'paper, as no support documentation has been provided, as is standard with and paper presentation) is that Spectracal have gone out of their way to promote it is a comparison of LS with CM, when the actual CM/LS comparison is a very small part of the overall paper.

I find that very disingenuous of Spectrcal, as they should hev gone out of their way to explain correctly exactly what the document contained.
It took a forum user to point out the CM actually can't have emulated data loading into it, so the majority of the document has no CM input at all.

After the document was release I personally contact Scott to ask for the data so we could run the same evaluation ourselves, and possible identify the issue (all be it basically invisible).

This was the main content of my mail to him:

"We do not believe there is any way LS can produce such noisy results, so are keep to replicate the test.
Can you please supply the data you used – the 21^3 emulation sets, as well as tell us how the noisy images were generated/rendered – via LS or eeColor – or some other way?

Either we can locate what has been done incorrectly with the test, or more likely locate an error within LS.
The results you have just don't add up with our own similar tests – so keen to get to the bottom of things."

Scott's reply totally ignored my request for the data:

"When looking back at the data set I it turns out I used the wrong patch set for the top plot of figure 7 so I need to correct that and send an updated report to spectracal. This will probably look more like what you would expect. The 21^3 interpolation errors shown in figure 8 and the summary table of the scaling are correct."

I attempted a follow-up, which has basically been ignored, and requested the same from Spectracal.

"To have validity we need to verify your tests, as with any normal scientific procedure.
Without that there is no truth.

Please forward your test data so we can replicate."

It is true, as has been pointed out in this thread, that the reported errors are basically invisible and of no consequence, but as we always strive for perfection we would have thought anyone attempting such 'evaluations' and then documenting the results, would participate in discussion as would the norm in any scientific test.

That has not been the case.

We will perform all our own testing, as it is possible missed error has crept in since we last performed the same tests, but having access to the data Scott used would have meant we could have proved the tests far more quickly and easily.

I personally thank the calibrators that have posting support of our position, even if they didn't really know or understand the reality of the situation, and are very glad such questions have been raised here.

We are less than impressed with Spectracal's use of material, that basically doesn't involve their product, as marketing hype.

Steve

I can confirm that there are major problems with Spectracal LUT building (Calman for Resolve). Starting from buggy interface to wrong reference levels.. had to spend several weeks with their superslow support, and had to use 3 different PC:s in order to get the first LUT done, and now it looks like there's no reliable way to verify and check how accurate the LUT is.. phew.

An update for you all...

After a bit of a delay, due to some urgent work for a major display manufacturer, we have at last got around to looking in detail at the evaluations Zoyd performed.
As per the message I sent him directly, and as I also posted in this thread, we have indeed found a 'rounding error' we managed to introduce into LightSpace some time back, and had missed.
If Zoyd (or Spectracal for that matter) had been polite enough to provide the test data Zoyd use we would have found this instantly...
But no matter, because as Zoyd stated. the error was basically invisible.

However, we strive for perfection, so were keen to understand what the error was.

So, a new version of LightSpace we will be releasing shortly (it's undergoing Beta testing at the moment).
This addresses the small issues noted.

There may be other rounding errors we have missed, so we are running more tests before we release this new version.
But, as above, the initial results seem to suggest the main (but invisible) issue has been solved.

Steve

This latest version of LightSpce has now been released.

Please update to it

Steve

So, Scott, the author of the original report has updated his findings using the latest version of LightSpace CMS.

We have also run our own test as well.

The results show Calman to be exceptionally poor in comparison to LightSpace...

Steve

For those of you that have followed this discussion, a number of independent test have recently been performed, ending with a direct comparison of LightSpace CMS and Calman on the same display, under exactly the same conditions, same probe, patch generator, etc...

Full details at: http://www.lightillusion.com/calibration_comparison.html

You can also see a lot of the new features in the next release of LightSpace - due any day!

Steve