It was 40 days ago when I mentioned the student license of TopSpin and other products, once pricey, that are now free for academic users. Knowing that many students prefer the MacBook and would like to run authentic Mac applications on it, I am going to write about the student promotion of iNMR. It is not free, but it is as cheap as it can possibly be. It's 39 euro (equivalent to 49 USD or 32 British pounds or 4200 Yens). What's so special about this license is that it is... perfectly normal! I mean: it INCLUDES direct customer support and this is quite valuable for a student that is learning NMR and a new software at the same time. Is this program difficult to learn? As for every NMR program, it CAN be hard if you are so familiar with TopSpin (or VNMR, or Jeol Delta) that you can't adapt yourself to anything else. The learning curve of iNMR is actually incredibly smooth if you start processing easy examples (1-D spectra or well acquired 2-D like TOCSY, HSQC...) before moving on to the esoteric.
Today you find video tutorials everywhere. The iNMR site offers "visual guides" instead. I feel more comfortable with the latter, first of all because English is not my native language; second of all because I can keep both the program and the guide open at the same time; third of all because I can read the guide at my own pace.
The iNMR manual is also worth of a mention. Actually I dedicated a whole post to it a few years ago. It is not the usual bulky PDF file. It looks like coming directly from Apple, because it closely resembles the manuals of Mail, Safari and iTunes for Mac OS X. Technically speaking, all these manuals are task-oriented. In simpler terms, every chapter answers to a question in the form: "I want to perform the operation X. I can I do it?". Here is an example. As you can see each chapter is just one page long.
In which cases would a student need help, then? An example is when she needs to write a script (a macro command); another extreme case is when she needs a modification to the program itself. Anyway, support also means giving a fast answer to people who can't find the time to read the manual. When you are a paying customer, you have your privileges.
For those who prefer freeware, there is the trial version of the same product (printing is disabled). iNMR has been around for 5 years by now, therefore there's plenty of reviews, short and long ones.
Thursday, 26 August 2010
Sunday, 22 August 2010
Software Engineering: The problem with the production line
Is software engineering the best approach for developing software? Does it apply for the majority of the software projects or just a very few of them?
Software Engineering was an answer for the perceived "software crisis", back in 1968, in the First NATO Software Engineering Conference and it was created to solve the problems of extremely large NATO and U.S. Department of Defence projects. In the majority of these projects, the hardware was still being designed and with no hardware to test, there was plenty of time to investigate requirements and write the software specifications. Hardware controlled by the software was generally worth billions of dollars like in the case of space shuttle and the Safeguard Ballistic Missile Defence System. People's lives and national security were also at stake.
The IEEE Computer Society's Software Engineering Body of Knowledge defines "software engineering" as:
Software Engineering for the Masses
More and more hardware became cheap and business of all sizes needed software in order to survive and be competitive. The difference was that a the big majority of these businesses couldn't afford to pay $35 million dollars per year and neither wait for too many years to start benefiting from their software. Also, many of those software projects were not manipulating expensive hardware or dealing with life-threatening situations. Edward Yourdon, in his book Rise and Resurrection of the American Programmer, wrote:
The "Good Enough Software" Era
Over the decades, many new software engineering processes and methodologies were created in order to make software development faster and cheaper. The most adopted were the ones based on iterative and incremental development that evolved into the agile software development.
Agile software development and "good enough software" were an amazing improvement in bringing costs down, mitigating risks with quicker feedbacks and much faster time to market.
Regardless the methodology or process used, software projects are still failing. Some fail because they were over-budget, others because they were not delivered on time, others failed to satisfy the requirements and business goals, et al.
The main problem is that for decades, software development was seen as a production line. That's the software engineering perspective of software development. Processes and methodologies are generally much more focused in making this production line more productive, creating different management and organisational styles than actually trying to make the employees more capable. Developers are treated as mere brain-damaged programmers and are at the bottom of the food chain.
Good enough is not always good enough
Using agile, lean or any methodology is not enough. It is all about the people involved in the project and their willingness to succeed and be proud of what they produce. If developers are seen as the least important and cheapest members of a software project, the maximum that this team is going to produce is mediocre software, regardless of methodology.
A software project would have much better chances to succeed with "good people with a bad process" than "mediocre people with a good process". Good people would always find a way to improve the process, be productive and produce something that they are proud of. Mediocre people accepts whatever is established, even when it is not good enough, and will produce just what they were asked for.
In a software project, if a company wants a good software, they will need good and empowered software developers. Instead of 10 mediocre people with a good process, it would be better to have 3 or 4 good people and empower them to deliver the project.
A process, imposed by managers and people that have no clue how to write software, will just guarantee that mediocre software is predictably delivered (if lucky). On the other hand, a team of self-organised and great developers would have a better shot at creating a more efficient way to produce great software, constantly trying to improve they way they work.
A process should never be more important than the people. Managers should facilitate the work of great software developers and not tell them what to do. It should always be harder to replace a good software developers than a manager since they are the ones that know the system inside-out. Managing a few well motivated, well paid and good professionals is always easier than manage many mediocre people.
Software development is a creative and highly skilled profession that takes years to master. While software development is treated like a production line, projects will continue to fail.
http://en.wikipedia.org/wiki/Software_engineering
http://en.wikipedia.org/wiki/Software_crisis
IEEE Standard Computer Dictionary, ISBN 1-55937-079-3, IEEE 1990
"They Write The Right Stuff", Fast Company, http://www.fastcompany.com/magazine/06/writestuff.html
Safeguard Program: http://en.wikipedia.org/wiki/Safeguard_Program
Stephenson, W. E. "An analysis of the resources used in the SAFEGUARD system software development"
Edward Yourdon - http://yourdon.com/about/
http://en.wikipedia.org/wiki/Iterative_and_incremental_development
http://en.wikipedia.org/wiki/Agile_software_development
Software Engineering was an answer for the perceived "software crisis", back in 1968, in the First NATO Software Engineering Conference and it was created to solve the problems of extremely large NATO and U.S. Department of Defence projects. In the majority of these projects, the hardware was still being designed and with no hardware to test, there was plenty of time to investigate requirements and write the software specifications. Hardware controlled by the software was generally worth billions of dollars like in the case of space shuttle and the Safeguard Ballistic Missile Defence System. People's lives and national security were also at stake.
The IEEE Computer Society's Software Engineering Body of Knowledge defines "software engineering" as:
Software engineering is the application of a systematic, disciplined, quantifiable approach to the development, operation, and maintenance of software, and the study of these approaches; that is, the application of engineering to software.Software Engineering can be very effective when developing safety critical systems like the one for the space shuttle as described on "They Write The Right Stuff" - Fast Company article, from 1996:
The last three versions of the program - each 420,000 lines long - had just one error each. The last 11 versions of this software had a total of 17 errors. Commercial programs of equivalent complexity would have 5,000 errors.Of course that this looks really impressive but there is more to it:
Money is not the critical constraint: The group's $35 million per year budget is a trivial slice of the NASA pie, but on a dollars-per-line basis, it makes the group among the nation's most expensive software organizations.Another extract from this same article when they were discussing the process:
And the culture is equally intolerant of creativity, the individual coding flourishes and styles that are the signature of the all-night software world. "People ask, doesn't this process stifle creativity? You have to do exactly what the manual says, and you've got someone looking over your shoulder," says Ted Keller (senior technical manager). "The answer is, yes, the process does stifle creativity."Many of the NATO and US Department of Defence took years, some over a decade to complete. Many had hundreds of people involved and almost half of the time was spend in requirements and design specifications, with uncountable reviews and approval cycles. Of course that, due to their reliability and quality, many are still in use today.
Software Engineering for the Masses
More and more hardware became cheap and business of all sizes needed software in order to survive and be competitive. The difference was that a the big majority of these businesses couldn't afford to pay $35 million dollars per year and neither wait for too many years to start benefiting from their software. Also, many of those software projects were not manipulating expensive hardware or dealing with life-threatening situations. Edward Yourdon, in his book Rise and Resurrection of the American Programmer, wrote:
I'm going to deliver a system to you in six months that will have 5,000 bugs in it - and you're going to be very happy!It was clear that software engineering processes had to be adapted in order to satisfy a more impatient and lower budget legion of businesses.
The "Good Enough Software" Era
Over the decades, many new software engineering processes and methodologies were created in order to make software development faster and cheaper. The most adopted were the ones based on iterative and incremental development that evolved into the agile software development.
Agile software development and "good enough software" were an amazing improvement in bringing costs down, mitigating risks with quicker feedbacks and much faster time to market.
Regardless the methodology or process used, software projects are still failing. Some fail because they were over-budget, others because they were not delivered on time, others failed to satisfy the requirements and business goals, et al.
The main problem is that for decades, software development was seen as a production line. That's the software engineering perspective of software development. Processes and methodologies are generally much more focused in making this production line more productive, creating different management and organisational styles than actually trying to make the employees more capable. Developers are treated as mere brain-damaged programmers and are at the bottom of the food chain.
Good enough is not always good enough
Using agile, lean or any methodology is not enough. It is all about the people involved in the project and their willingness to succeed and be proud of what they produce. If developers are seen as the least important and cheapest members of a software project, the maximum that this team is going to produce is mediocre software, regardless of methodology.
A software project would have much better chances to succeed with "good people with a bad process" than "mediocre people with a good process". Good people would always find a way to improve the process, be productive and produce something that they are proud of. Mediocre people accepts whatever is established, even when it is not good enough, and will produce just what they were asked for.
In a software project, if a company wants a good software, they will need good and empowered software developers. Instead of 10 mediocre people with a good process, it would be better to have 3 or 4 good people and empower them to deliver the project.
A process, imposed by managers and people that have no clue how to write software, will just guarantee that mediocre software is predictably delivered (if lucky). On the other hand, a team of self-organised and great developers would have a better shot at creating a more efficient way to produce great software, constantly trying to improve they way they work.
A process should never be more important than the people. Managers should facilitate the work of great software developers and not tell them what to do. It should always be harder to replace a good software developers than a manager since they are the ones that know the system inside-out. Managing a few well motivated, well paid and good professionals is always easier than manage many mediocre people.
Software development is a creative and highly skilled profession that takes years to master. While software development is treated like a production line, projects will continue to fail.
Source
Software Craftsmanship: The New Imperative - ISBN 0-201-73386-2, 2002 http://en.wikipedia.org/wiki/Software_engineering
http://en.wikipedia.org/wiki/Software_crisis
IEEE Standard Computer Dictionary, ISBN 1-55937-079-3, IEEE 1990
"They Write The Right Stuff", Fast Company, http://www.fastcompany.com/magazine/06/writestuff.html
Safeguard Program: http://en.wikipedia.org/wiki/Safeguard_Program
Stephenson, W. E. "An analysis of the resources used in the SAFEGUARD system software development"
Edward Yourdon - http://yourdon.com/about/
http://en.wikipedia.org/wiki/Iterative_and_incremental_development
http://en.wikipedia.org/wiki/Agile_software_development
Wednesday, 18 August 2010
Can Zero-Filling Correct the Baseline?
I want to show you a proton spectrum that has puzzled me during the last weeks. It contains something that's quite typical and something that I can't explain. I have processed the spectrum in two different ways, with zero-filling and without it. The spectrum without zero-filling is black, the spectrum with zero-filling is green (the number of points is doubled).
This detail is the bottom part of the TMS signal (magnified to show the ringing effect). Where does the ringing come from? TMS is a small symmetric molecule and its protons have a long relaxation time. Their signal persists at the end of the FID. When we add the zeroes after the signal, a step is created. The FT of the step is the ringing that we see. The spectrum without zero-filling doesn't contain the step, so there is no ringing. The period of the ringing is exactly 1 point. In simpler words: odd points are positive, even points are negative. Things are not so simple, actually, because the rule is reversed on the two sides of the peak. This is something I have always seen, I don't know if it's a constant rule or something that's merely more probable than its opposite.
Without zero-filling, we have only half of the points. They correspond to the maxima on the left of the peak and to the minima on the right of it. Any program for automatic phase correction is fooled by asymmetric peaks like this. Even humans are often fooled. They think that the spectrum is "difficult to phase" and don't recognize that the peak is asymmetric. Asymmetry and ringing are two sides of the same coin. Without zero-filling we have asymmetry, with zero-filling we have ringing. In the first case it is difficult to recognize that the signal is truncated, because it appears much larger than it actually is.
Up to this point I can explain everything. It's all familiar to me. There is another effect that I can't explain at all and appears when I observe the whole spectral range. The baseline of the normal spectrum is wavy.
The baseline is perfectly flat in the other case. This is the first time I see such an effect: can zero-filling correct the baseline?
This spectrum was acquired on a recent Jeol 400 MHz instrument. I wonder if the digital filter has anything to do with the latter effect.
This detail is the bottom part of the TMS signal (magnified to show the ringing effect). Where does the ringing come from? TMS is a small symmetric molecule and its protons have a long relaxation time. Their signal persists at the end of the FID. When we add the zeroes after the signal, a step is created. The FT of the step is the ringing that we see. The spectrum without zero-filling doesn't contain the step, so there is no ringing. The period of the ringing is exactly 1 point. In simpler words: odd points are positive, even points are negative. Things are not so simple, actually, because the rule is reversed on the two sides of the peak. This is something I have always seen, I don't know if it's a constant rule or something that's merely more probable than its opposite. Without zero-filling, we have only half of the points. They correspond to the maxima on the left of the peak and to the minima on the right of it. Any program for automatic phase correction is fooled by asymmetric peaks like this. Even humans are often fooled. They think that the spectrum is "difficult to phase" and don't recognize that the peak is asymmetric. Asymmetry and ringing are two sides of the same coin. Without zero-filling we have asymmetry, with zero-filling we have ringing. In the first case it is difficult to recognize that the signal is truncated, because it appears much larger than it actually is.
Up to this point I can explain everything. It's all familiar to me. There is another effect that I can't explain at all and appears when I observe the whole spectral range. The baseline of the normal spectrum is wavy.
The baseline is perfectly flat in the other case. This is the first time I see such an effect: can zero-filling correct the baseline?This spectrum was acquired on a recent Jeol 400 MHz instrument. I wonder if the digital filter has anything to do with the latter effect.
Friday, 6 August 2010
Which type of barista are you?
A colleague from New Zealand once was telling me about a type of coffee that was originated there. It's called Flat White Coffee. We were discussing about the difference between this coffee and all the other types of coffee with milk. Eventually we started talking about the quality of the drink and what makes it be better or worse. He mentioned that, of course, the quality of the beans and milk are very important for a good coffee but what makes a good coffee an excellent coffee is the barista's ability.
What makes a coffee to be a rubbish coffee then? If the coffee grains are rubbish, you will have a rubbish coffee. However, for a flat white coffee, coffee is just one variable in the equation. There are other variables like how the grains are roasted, steaming the milk at the right temperature, not adding sugar, how the milk is poured, the microfoam on top of the drink, etc. See the distinction from cafe con leche for details. Anyway, the point is, a rubbish (or careless) barista, with rubbish coffee will produce a rubbish coffee drink.
 |
| Rubbish Coffee |
 |
| Average Coffee |
 |
| Good Coffee |
 |
| Great Coffee |
Now imagine that you are the barista. But instead of coffee, you produce code. Imagine that the the coffee grains, milk and coffee machine are the tools you use to produce your code like the computer language, the IDE, the database, etc. Instead of serving coffee for your customers, you are producing software that will be used by your team mates, project sponsors, the company they (or you) work for, external clients, etc.
Like in the flat white coffee example above, of course that the tools we use are very important when producing a good software. However, more importantly, it is the quality of the software engineers that counts. A good barista can make a good coffee even when using average coffee grains, due to his or her ability to combine ingredients and prepare the drink. A bad barista can ruin the coffee even if he or she is using the best quality coffee beans. The biggest difference between the two baristas is how much they care about each cup of flat white coffee they prepare. Their pride and willingness to achieve the best pattern on top of each drink. The great feeling of achievement when they produce a great one. The happiness to see returning customers, queuing and waiting their turn to order the coffee that he, skilfully, prepares.
So, which type of barista are you?
Saturday, 17 July 2010
The Barrier
In the last two years NMR software has kept evolving slowly, but the world around it is changing more rapidly. From a technical point of view nothing important happened; while from a commercial point of view we are in the middle of a revolution. Prices have dropped down considerably. The false categories, the so-called "professional", "industry-standard" programs on a side and "low-budget", "alternative" programs on the other side, have disappeared. The companies that were selling the products in the first category have realized that the true values were actually reversed and adjusted the prices accordingly.
Consider TopSpin, for example. I remember its original price was in the range € 4000-5000. At the beginning of the month, I have visited an industrial lab where they had recently purchased a license for off-line processing. They told me they had paid € 3000 for TopSpin. They had also considered a solution by ACDlabs, but after receiving a quote of € 10,000 they renounced with disgust. The main reason why they prefer TopSpin is that it is possible to process the spectra directly on the spectrometer, with great saving of time. When back into the lab, they simply print the already processed spectra.
Less than 24 hours later I had a pleasant talk with Angelo Ripamonti, from Bruker Italy. He gave me different figures. The price of a license is € 2,000, complete with box, cartaceous manual and CD. They also have an electronic edition, called "Topspin Student Edition". It is the same product, the license expires after 3 years and costs € 99 (as far as I understand, there is no after-sale support; I am not sure, though). With my great surprise, Angelo said that they hope the students will familiarize with TopSpin during their PhD period and remain faithful to it. I used to think that TopSpin was by far the most famous of all the NMR programs; evidently they are not so sure and fear the competition.
On one thing we agreed all along the line: the chemical industry has completely disappeared from Italy (if we are allowed to measure it by the number of magnets being sold). If I could, I would also close the Italian universities (what's its purpose, if there is no industry? the local job market wants nurses, not chemists), but that's another story (Angelo hopes the Universities will grow and buy more magnets).
I told Angelo that I had noticed that ACD had their own "academic edition", which is completely free. I myself have received a copy, though never used yet. I asked for his opinion: could the ACD move be a reply to Bruker's Student Edition? "No", he said, "they are making a lot of money from their NMR database". Translation: ACD is the only competitor in the field of NMR database and they want to monetize as much as they can while the favorable situation persists. Indeed, the "price" I paid for the NMR processor was my email address, which they have already used to advertise the database.
Bruker is also lured by this segment of the market. They bought Perch and are working on it to make a new product that will predict the chemical shifts from the structure. Not the same thing as a database, but with the same purpose.
As my readers know, I had long waited to try the ACD program, because I started this blog with the intention of writing reviews for all the software in existence. Time passed and I have different pastimes today. If I haven't found the time yet to study the Processor and write a review, there are very few chances I can do it in future. The mere length of the manual discourages me. I want however to comment on their commercial move.
(1) It is not correct to first sell a program to several universities and then to give it for free to the rest of the world. Unless they give the database for free to those universities that paid for the processor. That would be a fair compensation. What about, however, the universities who already have bought both products? What about those who never bought anything? After this precedent, how can ACD convince somebody to pay for one of their products, when there's the risk that it becomes free after a few years?
(2) People see with different eyes a program that is free starting from version 1 and a program that is free starting from version 12. Consider, for example, Internet Explorer. It was born as a freeware and it was a huge success, actually a monopoly. It still accounts for 45% of the market. Firefox accounts for the 32%. Opera, instead, started as a commercial product before becoming free. Its share of the market is a disappointing 2%. What people feel is that if the former commercial product was not good enough to sell copies, it is not good enough to bother with.
(3) I don't know exactly why, but ACD has not been lucky, so far, with free programs. Consider their ChemSketch, for example. I have never found a bad review, while I always hear people complaining about ChemDraw. Somehow the latter remains the undisputed no. 1 in the field. Why?
(4) If you want your product to become popular, making it free is a good move, but giving it a catchy name is as much as important. People at ACD have always lacked in inspiration. "ACD NMR Processor Academic Edition" is the less memorable of the names. Even a distasteful name like "BottomSpin" would have proved more effective.
(5) If you analyze the situation in detail, this product is still far from being "free". Whenever you print, the program adds a red line reminding it's not to be used for commercial purposes. The file format is proprietary and not recognized by all the programs (you are locked-in). There is a potential risk that the program becomes commercial again in future.
The battle has reached the first result. It has created a barrier. If a new competitor wants to enter into the arena, its product must be at least as good as today's free programs (and there are at least 3 formerly commercial programs that are available for free today). Given that it costs a lot of time and money to create a new NMR program, while the prices keep falling, the barrier is very high. We are not going to see any novel program in the next decade, neither free nor commercial. If all the investments are made in the field of databases and predictors, even existing processing programs are not going to be revamped, but simply refreshed.
Do not think that users are happy and programmers are sad. Not at all! After 4 years, the most read page of this blog remains "TopSpin Free Download" and readers who comments there are angry as before (the page exists, but there is nothing to download!). All the programmers I am talking with, on the other hand, are quite glad: their programs are selling and nobody went in bankrupt in the last few years, despite the financial crises. Apparently a point of contact has been found between the two sides. The prices are more reasonable than they used to be a decade ago, the programs are better and many customers prefer to pay if they can receive a good service in addition to the product.
Consider TopSpin, for example. I remember its original price was in the range € 4000-5000. At the beginning of the month, I have visited an industrial lab where they had recently purchased a license for off-line processing. They told me they had paid € 3000 for TopSpin. They had also considered a solution by ACDlabs, but after receiving a quote of € 10,000 they renounced with disgust. The main reason why they prefer TopSpin is that it is possible to process the spectra directly on the spectrometer, with great saving of time. When back into the lab, they simply print the already processed spectra.
Less than 24 hours later I had a pleasant talk with Angelo Ripamonti, from Bruker Italy. He gave me different figures. The price of a license is € 2,000, complete with box, cartaceous manual and CD. They also have an electronic edition, called "Topspin Student Edition". It is the same product, the license expires after 3 years and costs € 99 (as far as I understand, there is no after-sale support; I am not sure, though). With my great surprise, Angelo said that they hope the students will familiarize with TopSpin during their PhD period and remain faithful to it. I used to think that TopSpin was by far the most famous of all the NMR programs; evidently they are not so sure and fear the competition.
On one thing we agreed all along the line: the chemical industry has completely disappeared from Italy (if we are allowed to measure it by the number of magnets being sold). If I could, I would also close the Italian universities (what's its purpose, if there is no industry? the local job market wants nurses, not chemists), but that's another story (Angelo hopes the Universities will grow and buy more magnets).
I told Angelo that I had noticed that ACD had their own "academic edition", which is completely free. I myself have received a copy, though never used yet. I asked for his opinion: could the ACD move be a reply to Bruker's Student Edition? "No", he said, "they are making a lot of money from their NMR database". Translation: ACD is the only competitor in the field of NMR database and they want to monetize as much as they can while the favorable situation persists. Indeed, the "price" I paid for the NMR processor was my email address, which they have already used to advertise the database.
Bruker is also lured by this segment of the market. They bought Perch and are working on it to make a new product that will predict the chemical shifts from the structure. Not the same thing as a database, but with the same purpose.
As my readers know, I had long waited to try the ACD program, because I started this blog with the intention of writing reviews for all the software in existence. Time passed and I have different pastimes today. If I haven't found the time yet to study the Processor and write a review, there are very few chances I can do it in future. The mere length of the manual discourages me. I want however to comment on their commercial move.
(1) It is not correct to first sell a program to several universities and then to give it for free to the rest of the world. Unless they give the database for free to those universities that paid for the processor. That would be a fair compensation. What about, however, the universities who already have bought both products? What about those who never bought anything? After this precedent, how can ACD convince somebody to pay for one of their products, when there's the risk that it becomes free after a few years?
(2) People see with different eyes a program that is free starting from version 1 and a program that is free starting from version 12. Consider, for example, Internet Explorer. It was born as a freeware and it was a huge success, actually a monopoly. It still accounts for 45% of the market. Firefox accounts for the 32%. Opera, instead, started as a commercial product before becoming free. Its share of the market is a disappointing 2%. What people feel is that if the former commercial product was not good enough to sell copies, it is not good enough to bother with.
(3) I don't know exactly why, but ACD has not been lucky, so far, with free programs. Consider their ChemSketch, for example. I have never found a bad review, while I always hear people complaining about ChemDraw. Somehow the latter remains the undisputed no. 1 in the field. Why?
(4) If you want your product to become popular, making it free is a good move, but giving it a catchy name is as much as important. People at ACD have always lacked in inspiration. "ACD NMR Processor Academic Edition" is the less memorable of the names. Even a distasteful name like "BottomSpin" would have proved more effective.
(5) If you analyze the situation in detail, this product is still far from being "free". Whenever you print, the program adds a red line reminding it's not to be used for commercial purposes. The file format is proprietary and not recognized by all the programs (you are locked-in). There is a potential risk that the program becomes commercial again in future.
The battle has reached the first result. It has created a barrier. If a new competitor wants to enter into the arena, its product must be at least as good as today's free programs (and there are at least 3 formerly commercial programs that are available for free today). Given that it costs a lot of time and money to create a new NMR program, while the prices keep falling, the barrier is very high. We are not going to see any novel program in the next decade, neither free nor commercial. If all the investments are made in the field of databases and predictors, even existing processing programs are not going to be revamped, but simply refreshed.
Do not think that users are happy and programmers are sad. Not at all! After 4 years, the most read page of this blog remains "TopSpin Free Download" and readers who comments there are angry as before (the page exists, but there is nothing to download!). All the programmers I am talking with, on the other hand, are quite glad: their programs are selling and nobody went in bankrupt in the last few years, despite the financial crises. Apparently a point of contact has been found between the two sides. The prices are more reasonable than they used to be a decade ago, the programs are better and many customers prefer to pay if they can receive a good service in addition to the product.
Thursday, 1 July 2010
Universal Hole
Earlier in this week I cited a paper by Kobzar and Luy. It contains the statement:
that confirms and enforces what I have always being saying:
What they have found is a kind of super-hole that is common to every program. My first thought would normally be: "If nobody cares, why should I?", but this time I was intrigued by a figure just above the cited statement. That figure resembles a picture of mine I published here a few months ago. Despite the apparent similarity, however, the two methods have little in common.
Driven by curiosity, I looked on the web for anything more recent on the same subject and found this page that describes the very same "long range J" procedure. Does it mean that somebody has already filled the hole?
I have contacted the PR man at nucleomatica and he explained that the procedure is not commercial yet. It is a very simple data manipulation, there is no secret about it, but neither there is demand for it by the market. In conclusion, there is no hurry to make it available (to a distracted public).
Finally he gave me this picture, which is a world-exclusive of my blog:
Believe it or not, what you see is the same multiplet shown into the JMR figure (page 133, fig. 3d). Same molecule, same kind of experiment, another sample, another instrument.
The two experimental multiplets in black differ for the absence (top) or presence (bottom) of an anti-phase coupling. Both traces come from 2-D experiments and the resolution can never be enough to directly measure the size of the coupling.
The green circle hilights a slider. When you move the slider, the program adds an artificial coupling to the upper trace. The result is shown in red. When the red multiplet is like the black multiplet at the bottom we have succesfully simulated the missing coupling AND NOW WE KNOW HOW LARGE IT IS. That's what it's all about.
If you remember, I have gone much further with my unbeatable simulator, because it is able to extract all the couplings with a single experiments.
The strenght of my method is, alas, also its drawback: even when you are interested into a single J value, you are forced to measure them all. It can be very hard in cases like this.
Despite the external similarities the two methods are very different inside, serve two different purposes and can live side by side very well into the same program.
The coupling extraction procedure is not yet implemented in any available software.
that confirms and enforces what I have always being saying:
Any NMR program contains some hole and by the time it's filled another hole appears.
What they have found is a kind of super-hole that is common to every program. My first thought would normally be: "If nobody cares, why should I?", but this time I was intrigued by a figure just above the cited statement. That figure resembles a picture of mine I published here a few months ago. Despite the apparent similarity, however, the two methods have little in common.
Driven by curiosity, I looked on the web for anything more recent on the same subject and found this page that describes the very same "long range J" procedure. Does it mean that somebody has already filled the hole?
I have contacted the PR man at nucleomatica and he explained that the procedure is not commercial yet. It is a very simple data manipulation, there is no secret about it, but neither there is demand for it by the market. In conclusion, there is no hurry to make it available (to a distracted public).
Finally he gave me this picture, which is a world-exclusive of my blog:
Believe it or not, what you see is the same multiplet shown into the JMR figure (page 133, fig. 3d). Same molecule, same kind of experiment, another sample, another instrument.The two experimental multiplets in black differ for the absence (top) or presence (bottom) of an anti-phase coupling. Both traces come from 2-D experiments and the resolution can never be enough to directly measure the size of the coupling.
The green circle hilights a slider. When you move the slider, the program adds an artificial coupling to the upper trace. The result is shown in red. When the red multiplet is like the black multiplet at the bottom we have succesfully simulated the missing coupling AND NOW WE KNOW HOW LARGE IT IS. That's what it's all about.
If you remember, I have gone much further with my unbeatable simulator, because it is able to extract all the couplings with a single experiments.
The strenght of my method is, alas, also its drawback: even when you are interested into a single J value, you are forced to measure them all. It can be very hard in cases like this.
Despite the external similarities the two methods are very different inside, serve two different purposes and can live side by side very well into the same program.
Tuesday, 29 June 2010
Faster Faster Faster
Our machines, even when hitting an apparent performance peak, only run at one small fraction of their true potential speed. I feel that today's computers and their software are OK for routine spectra. I couldn't ask for more. Other spectra are quite large, however, and I must wait a few seconds during processing. Without going into the third dimension, consider these novel experiments to measure long range heteronuclear Js. Each row contains at least 4096 points. Quite likely we are going to see larger rows in the next few years. The time required to compute the FFT is in the order of the seconds. It would be great if we could half this time. The solution is public since 2008 and freely available. It must also be well-know, because I have initially found it on Wikipedia.
What they say, in practice, is that if you use the GPU (the graphic chip) instead of the CPU (the main brain of the computer)...
Another paper says:
The source code (to be compiled), is available on another site. What upsets me is the ReadMe file:
Maybe they have already found the solution to this problem.
There's another issue, however: how long does it take to move the matrix from the main memory to the GPU and back? I presume that loading the columns will take much more time than loading the rows. In this case it is better to transpose the matrix between the two FFTs (just like when we use the CPU). Eventually, the bottleneck will the the transposition, not the FT.
What they say, in practice, is that if you use the GPU (the graphic chip) instead of the CPU (the main brain of the computer)...
In this work we present a novel implementation of FFT on GeForce 8800GTX that achieves 144 Gflop/s that is nearly 3x faster than best rate achieved in the current vendor’s numerical libraries.
Another paper says:
We implemented our algorithms using the NVIDIA CUDA API and compared their performance with NVIDIA's CUFFT library and an optimized CPU-implementation (Intel's MKL) on a high-end quad-core CPU. On an NVIDIA GPU, we obtained performance of up to 300 GFlops, with typical performance improvements of 2--4x over CUFFT and 8--40x improvement over MKL for large sizes.
The source code (to be compiled), is available on another site. What upsets me is the ReadMe file:
Currently there are a few known performance issues (bug) that this sample has discovered in rumtime and code generation that are being actively fixed. Hence, for sizes >= 1024, performance is much below the expected peak for any particular size. However, we have internally verified that once these bugs are fixed, performance should be on par with expected peak. Note that these are bugs in OpenCL runtime/compiler and not in this sample.
Maybe they have already found the solution to this problem.
There's another issue, however: how long does it take to move the matrix from the main memory to the GPU and back? I presume that loading the columns will take much more time than loading the rows. In this case it is better to transpose the matrix between the two FFTs (just like when we use the CPU). Eventually, the bottleneck will the the transposition, not the FT.
Subscribe to:
Posts (Atom)