Post by Ron Brinkmann
Cross Posted at DigitalComposting.com
I’ve been saying for a long time now that cameras need to evolve to where they’re an open computing platform. To where all of the hardware on the device can be programmatically controlled by onboard software applications. Unfortunately we haven’t seen a whole lot of movement in this area from the big camera manufacturers, other than a bit of SDK support by Canon and (finally) Nikon and some interesting but cumbersome hackish options.
I know that part of the reason for this is that the software/firmware on a camera isn’t really designed with this in mind – it’s not necessarily an easy change to develop an architecture that would support 3rd party applications. Which is why I’m starting to think that this will end up being solved in the other direction – by dedicated computing platforms that also happen to have camera capabilities. Platforms like, for instance, the iPhone.
Now clearly the title of this post is intended to be a bit ridiculous. I’ll be the first in line to talk about how crap the current iPhone camera is. But the limitations are primarily due to the hardware. And camera hardware, at least up to a certain point, is a really cheap commodity.
So let’s talk a little bit about what I could do if I had a device with decent camera hardware (a reasonably good sensor, onboard flash, maybe some other worthwhile stuff like GPS, WiFi, etc.) along with full access to that hardware via a real programming interface like the one available for the iPhone. Here’s just a few ideas:
-Timelapse. This is such a simple feature from a software perspective yet many (probably most) cameras don’t support it. For my DSLR I need to buy an external trigger with timer capabilities. Why? (Other than the obvious answer, which is because it allows the camera manufacturers to charge a ridiculous amount of money for such a remote trigger. Hint – buy an ebay knockoff instead).
-Motion detection. It’s a simple enough algorithm to detect movement and only trigger a photo when that occurs. Yes, this would make it easy to set up a camera to see if the babysitter or the house-painters wander into rooms they shouldn’t, but it would also be very interesting to use with, for example, wildlife photography.
-Allow me to name files sensibly and according to whatever scheme I want. Right now I rename my images as soon as they’re loaded onto my computer (based on the day/time the photo was taken) but why can’t my camera just do that for me automatically? And do it according to any scheme I care to specify.
-Let me play with the shutter/flash timing. Allow me to do multiple-flashes over the duration of a long exposure, for instance, to get interesting multiple-exposure effects.
-Give me programmatic control over the autofocus and the zoom (if it’s servo-controlled), so I can shoot bracketed focus-points or animate the zoom while the shutter is open for interesting effects.
-Overlay a reticle on the image, so I can see the framing on different aspect ratios before I shoot the photo.
-Add a nice touchscreen display to the camera and then I’ll be able to easily choose my focus point(s) or choose a region of the image that I want to meter on.
-Activate the microphone to do a voice annotation or capture some ambient sound with every photo. Or for that matter, allow me to voice-activate the shutter. (There’s a whole world full of sound-activation possibilities I suspect…)
-Allow me to reprogram the buttons and dials on the camera in any way I see fit. Any of the buttons/dials, not just a couple of special-case ones…
Some of this hardware control could go down to a very deep level. Let me play with the scanning rate of the rolling shutter, for instance, to give me super jello-cam images.
And then there’s a whole huge variety of general image-processing operations that could be applied in the camera, from custom sharpening algorithms to specialized color-corrections to just about anything else you currently need to do on your computer instead. HDR image creation. Tourist Removal. Etc. Are some of these better done as post-processes rather than in-camera? Sure, probably. But you could make the same claim about a lot of things – it’s why some people shoot RAW and others are happy with JPEG. Bottom line is that there are times where you just want to get a final photo straight out of the camera.
(of course the cool thing would be to have the camera create the processed JPEG while also storing the original RAW file. And then output a metadata file that, when read into Aperture or Lightroom or whatever your RAW processing package of choice is, would duplicate the image processing operations you’d applied in-camera but then allow you to add/remove/modify as you see fit).
And if your camera includes additional useful hardware like GPS or network access or accelerometers then you’ll have even more functionality you can access. Take a look at the iphone app Night Camera which uses the accelerometer to measure when the camera reaches a relatively stable position before taking a photo. A really smart idea and one that will almost certainly show up in other ‘real’ cameras relatively soon. (Incidentally, it’s worth noting that the main difficult with algorithmically removing motion blur caused by camera shake is due to the uncertainty about exactly how the camera moved while the photo was being taken – if we were to record accurate accelerometer readings over the duration of the time that the shutter was open, it becomes much easier to remove the blur as a post-process).
Now, before I get a bunch of replies stating “my camera can already do that” on some of the items listed above, let me reiterate my original point – a camera with the ability to run arbitrary camera-control software would be able to do all of the above items. And a lot more. The real fun will start when when lots of people are brainstorming about this and then churn out features that nobody has thought of yet. Who knows what this might include?
Some of these feature may not feel terribly important or may even seem ‘gimmicky’, but all it takes is a single special-case situation where you need one of them and you’ll be glad they’re part of your software arsenal.
Back to the iPhone, it’s worth mentioning that we’re already seeing some of these sorts of applications show up. There are panoramic stitchers and the aforementioned accelerometer function and multiple-exposure resolution enhancers and even a very nice timelapse application. We may also end up seeing a sort of hybrid solution, with a portable device like the iPhone is tethered to the camera to give the desired capabilities.
I’m convinced that this is where the future of the camera is going, at least from a software perspective* and the features listed above are just a small subset of what we’ll end up seeing. But you tell me – If you had full software control over your camera, what would you do with it?
*The future of the camera from a hardware perspective is a whole other ball of wax but I’ll save my thoughts on that for a later post. For now I’ll just say that I’m betting that the current paradigm of single (expensive) lens + single (expensive) sensor will eventually be replaced by multiple lenses and sensors and some sophisticated stitching software. Some sort of Redundant Array of Inexpensive Lenses and Sensors (I call it RAILS). I’ll try to get a post together on that topic as soon as I get a chance.
Pingback: TWIPPHOTO.COM » iPhone: The Perfect Camera? - TWIPPHOTO.COM | Software, Computers...
Pingback: TWIPPHOTO.COM » iPhone: The Perfect Camera? - TWIPPHOTO.COM | Software, Computers...
Interesting thoughts Ron!
I think it has to be the way forward with the software side of things. The iPhone has really set the standard with expectations of new gadgetry and I hope the camera manufacturers are seriously looking into incorporating these functions into their future designs.
A ‘Wobble’ function on a D3x would be quite sublime…..
James
Freiburg, Germany
This isn’t one of those “my camera can already do that” replies, though I suppose it’s close. I haven’t got an Iphone, I do however have a Nokia N95, it’s 5 schmegapixels, can actually autofocus and has a flash. It isn’t a fantastic camera by any stretch of the imagination but it is always in my pocket and so sees as much use as my dslr. It has also got an accelerometer, GPS and wifi and as far as I know it is open for anyone to write applications for.
I am aware that a lot of people have what I refer to as Iphone-itis, an affliction resulting in a complete inability to recognize other smartphones on the market, but despite its futility I’d like to remind you all that there are alternatives, especially if you are interested in using the phone for photography.
The Iphone does have one thing a lot of technically superior phones do lack however, the application store. This I imagine makes it more likely for really sophisticated software, software that you’d really have to charge for, to be available for that platform.
This isn’t one of those “my camera can already do that” replies, though I suppose it’s close. I haven’t got an Iphone, I do however have a Nokia N95, it’s 5 schmegapixels, can actually autofocus and has a flash. It isn’t a fantastic camera by any stretch of the imagination but it is always in my pocket and so sees as much use as my dslr. It has also got an accelerometer, GPS and wifi and as far as I know it is open for anyone to write applications for.
I am aware that a lot of people have what I refer to as Iphone-itis, an affliction resulting in a complete inability to recognize other smartphones on the market, but despite its futility I’d like to remind you all that there are alternatives, especially if you are interested in using the phone for photography.
The Iphone does have one thing a lot of technically superior phones do lack however, the application store. This I imagine makes it more likely for really sophisticated software, software that you’d really have to charge for, to be available for that platform.
I have come to hate the Canon file naming system. i cant figure out how to change the file to make a new one every day I take pictures. That would be my preference. I really like the 40D/Iphone composite.
I have come to hate the Canon file naming system. i cant figure out how to change the file to make a new one every day I take pictures. That would be my preference. I really like the 40D/Iphone composite.
Good post, Ron. You make a very good case for allowing proper custom software control of digital cameras. It’s not a subject I’ve seen tackled like this before and it is very thought provoking.
It has to come someday, but I’m not sure from which direction. The established camera brands do innovate but are evolutionary rather than revolutionary in the way they move forward from familiar paradigms. I guess most keen amateurs wouldn’t go into this territory – it will be pressure from the very techy professionals and the geekier amateurs.
The software capability is kinda already there at the cellphone end but when are you going to get a cellphone camera that can compete quality-wise with the high-end DSLR of the day?
We’ll get there but don’t hold your breath.
Good post, Ron. You make a very good case for allowing proper custom software control of digital cameras. It’s not a subject I’ve seen tackled like this before and it is very thought provoking.
It has to come someday, but I’m not sure from which direction. The established camera brands do innovate but are evolutionary rather than revolutionary in the way they move forward from familiar paradigms. I guess most keen amateurs wouldn’t go into this territory – it will be pressure from the very techy professionals and the geekier amateurs.
The software capability is kinda already there at the cellphone end but when are you going to get a cellphone camera that can compete quality-wise with the high-end DSLR of the day?
We’ll get there but don’t hold your breath.
I think the problem that the other devices have basically revolves around mind share and marketing. Up until the iPhone introduction, smart phones always had this perception or stigma that it’s either a business tool or a geek’s toy. Not only that, Nokia smart phones have a low profile in the US, the one I’ve seen on display was just a plastic dummy. I’m pretty wary of things that I can’t try out before buying. It would be nice if the screen were larger too.
I think the problem that the other devices have basically revolves around mind share and marketing. Up until the iPhone introduction, smart phones always had this perception or stigma that it’s either a business tool or a geek’s toy. Not only that, Nokia smart phones have a low profile in the US, the one I’ve seen on display was just a plastic dummy. I’m pretty wary of things that I can’t try out before buying. It would be nice if the screen were larger too.
I like a lot of those ideas a lot. Not that I would use everyone of them, but I can see how somebody could have a need and I am sure that there are many more ideas that could be realized.
My idea: One of my biggest problems with my DSLR (which is my first SLR, too) is mastering depth of field. Sure – I read about focal length and f-stops and all of that… but the formulas to calculate depth of field are a bit heavy to be used without a calculator and my first DSLR had no depth of field preview. Why not have a function in the DSLR that’ll give you the depth of field of view for the given settings? It could work on DSLRs without a depth of field preview or when the light is so low, that it the depth of field is hard to judge with the preview button.
I think that the reasons why we don’t have something like that already are not really of a technical nature. My goodness: my mobile has more computing power and the 4 GB card in my DSLR has more than 100 times more storage than my first computer. I think the problem is more of a marketing and/or psychological nature.
At first, companies need to realize that there is a market for this kind of thing. And then they’ll need to figure out a strategy for how to present to the customer that this is something worthwhile for them.
When I got my first DSLR, it totally baffled me how much companies still concentrated on marketing the pixel count. I mean, come on! I was as wet behind the ears as you can get (some might say I still are) and even I realized that every camera available in that market niche had more pixels than I or most amateurs are likely to use. I mean realistically – how many of us print larger than A3 on a regular basis? Shooting with a SLR – wether it is digital or not – needs somewhat more knowledge than shooting with a point and shoot. I could get why people who prefer a compact point and shoot might fall for the megapixel marketing strategies – but people looking for a DSLR? Was I overestimating DSLR shooters or were the companies underestimating them? I think – and I might be mistaken – that it is the later. And as long as the big companies don’t realize that their customers are more tech-savvy than they gave them credit for, I don’t see customizable camera software anytime soon in the future. Which is a pity really, because existing hardware could do so much more…
I like a lot of those ideas a lot. Not that I would use everyone of them, but I can see how somebody could have a need and I am sure that there are many more ideas that could be realized.
My idea: One of my biggest problems with my DSLR (which is my first SLR, too) is mastering depth of field. Sure – I read about focal length and f-stops and all of that… but the formulas to calculate depth of field are a bit heavy to be used without a calculator and my first DSLR had no depth of field preview. Why not have a function in the DSLR that’ll give you the depth of field of view for the given settings? It could work on DSLRs without a depth of field preview or when the light is so low, that it the depth of field is hard to judge with the preview button.
I think that the reasons why we don’t have something like that already are not really of a technical nature. My goodness: my mobile has more computing power and the 4 GB card in my DSLR has more than 100 times more storage than my first computer. I think the problem is more of a marketing and/or psychological nature.
At first, companies need to realize that there is a market for this kind of thing. And then they’ll need to figure out a strategy for how to present to the customer that this is something worthwhile for them.
When I got my first DSLR, it totally baffled me how much companies still concentrated on marketing the pixel count. I mean, come on! I was as wet behind the ears as you can get (some might say I still are) and even I realized that every camera available in that market niche had more pixels than I or most amateurs are likely to use. I mean realistically – how many of us print larger than A3 on a regular basis? Shooting with a SLR – wether it is digital or not – needs somewhat more knowledge than shooting with a point and shoot. I could get why people who prefer a compact point and shoot might fall for the megapixel marketing strategies – but people looking for a DSLR? Was I overestimating DSLR shooters or were the companies underestimating them? I think – and I might be mistaken – that it is the later. And as long as the big companies don’t realize that their customers are more tech-savvy than they gave them credit for, I don’t see customizable camera software anytime soon in the future. Which is a pity really, because existing hardware could do so much more…
It wouldn’t surprise me at all if the ‘big guys’, i.e. canon+nikon, get outmaneuvered by a smaller, more nimble player who comes in from a radically different angle. It’s not likely to happen with a DSLR – too much expertise to replicate. The game-change will happen with something that is more mass-market consumer-focused I’d bet.
It wouldn’t surprise me at all if the ‘big guys’, i.e. canon+nikon, get outmaneuvered by a smaller, more nimble player who comes in from a radically different angle. It’s not likely to happen with a DSLR – too much expertise to replicate. The game-change will happen with something that is more mass-market consumer-focused I’d bet.
Speaking of printing, I wonder how many folks are like me. Except for a few wallet sized prints of the grandkids and some Moo cards with my flickr address, I don’t print ANY of my work. It is all web based. Five or six MP anyone?
Speaking of printing, I wonder how many folks are like me. Except for a few wallet sized prints of the grandkids and some Moo cards with my flickr address, I don’t print ANY of my work. It is all web based. Five or six MP anyone?
I think the title of the post should be:
“Andriode or Nokia (Maemo) phone: The Perfect Camera?”
While the iPhone has 3rd party applications, it is still a quite closed environment, as Apple has veto power over when 3rd party applications run on the iPhone. If your app duplicates a function that is already there Apple vetos it. On a camera many 3rd party apps I can see being attractive to photographers would be ones that give you more finer controls over the existing settings, so they operate in a more techy/geeky/pro way.
I agree with Ron’s thesis I just think that the iPhone is a a good example of it.
I think the title of the post should be:
“Andriode or Nokia (Maemo) phone: The Perfect Camera?”
While the iPhone has 3rd party applications, it is still a quite closed environment, as Apple has veto power over when 3rd party applications run on the iPhone. If your app duplicates a function that is already there Apple vetos it. On a camera many 3rd party apps I can see being attractive to photographers would be ones that give you more finer controls over the existing settings, so they operate in a more techy/geeky/pro way.
I agree with Ron’s thesis I just think that the iPhone is a a good example of it.
There’s a a piece of software available for the G7 (and anything that uses Canon’s Digic II & Digic III) that enables a surprising amount of this, including time lapse and motion detection and the ability to save RAW. It’s called CHDK and acts as alternative firmware that can be optionally loaded each time the camera starts.
I agree I’d love to see it more officially supported and therefore become available for even better cameras, but you might want to check it out.
There’s a a piece of software available for the G7 (and anything that uses Canon’s Digic II & Digic III) that enables a surprising amount of this, including time lapse and motion detection and the ability to save RAW. It’s called CHDK and acts as alternative firmware that can be optionally loaded each time the camera starts.
I agree I’d love to see it more officially supported and therefore become available for even better cameras, but you might want to check it out.
Back in the early days of digital cameras, the Kodak DC290 and earlier versions had the capability of being programmed with the Digita Scripting language. You could write or download and run scripts in the camera for things like time lapse photography. I guess it never caught on commercially because Kodak stopped putting in their cameras but I was thought it was great!
There are some really interesting thoughts there Ron.
With resepect to the voice annotation with each shot, a text-to-speach processor would make this even more handy. Imagine being able to at the start of a shoot say into the mocrophone “I’m in Washington shooting President Obama’s Innaguration Ceremony” and have the meta-data of all the days shots tagges with that. Would save so much time keywording on import (and maybe even name the photos appropriately).
There are some really interesting thoughts there Ron.
With resepect to the voice annotation with each shot, a text-to-speach processor would make this even more handy. Imagine being able to at the start of a shoot say into the mocrophone “I’m in Washington shooting President Obama’s Innaguration Ceremony” and have the meta-data of all the days shots tagges with that. Would save so much time keywording on import (and maybe even name the photos appropriately).
Interesting – I wasn’t aware of this. Pretty cool. What year was that?
Interesting – I wasn’t aware of this. Pretty cool. What year was that?
Yeah, there was a link to the CHDK hack (and to several other supplemental bits of info) in the original post I put up on my blog but those links all got truncated when this was crossposted here. Click on the link at the top of the page if you want to see them.
There is a lot of additional functionality in CHDK but the implementation is pretty user-unfriendly – which goes back to how much better a general-purpose framework would be for this sort of thing…
Yeah, there was a link to the CHDK hack (and to several other supplemental bits of info) in the original post I put up on my blog but those links all got truncated when this was crossposted here. Click on the link at the top of the page if you want to see them.
There is a lot of additional functionality in CHDK but the implementation is pretty user-unfriendly – which goes back to how much better a general-purpose framework would be for this sort of thing…
Pingback: The perfect camera | Light and Pixels
Pingback: The perfect camera | Light and Pixels
Great article. I definitely agree with the premise of the article. It seems there is a great deal of creativity and innovation that is being missed by not making greater use of the flexibility the software allows. There are several functions Ron mentions that seem like they should require only software changes to make. The lack intervalometers (desite showing up in some P&S cameras) and the lack of fully programmable buttons are my personal peeves. It would be a huge improvement to even have a portion of the software source available to the public to post their own modifications; even if applications were made available for sale (say $10) I’d imagine there is a great deal of money to made because of the shear number of people who’d buy them.
Great article. I definitely agree with the premise of the article. It seems there is a great deal of creativity and innovation that is being missed by not making greater use of the flexibility the software allows. There are several functions Ron mentions that seem like they should require only software changes to make. The lack intervalometers (desite showing up in some P&S cameras) and the lack of fully programmable buttons are my personal peeves. It would be a huge improvement to even have a portion of the software source available to the public to post their own modifications; even if applications were made available for sale (say $10) I’d imagine there is a great deal of money to made because of the shear number of people who’d buy them.
Ron … From your lips to God’s ears.
Ron … From your lips to God’s ears.
I have the Canon 40D. It has both a DOF preview and live view.
The DOF preview button allows you to stop down to the set aperture and see the actual DOF in the viewfinder. Too dark? Too small?
Then combine it with the liveview mode (combined with show true exposure for live view). When i push the DOF preview button in live view, it will calculate exposure for the given settings – you can see both the exposure and the DOF, live.
My point is that whatever photographers can think up that they want in their cameras, you can bet the camera companies have thought of it as well. They are just bound by marketing – they will sell whatever sells now, until it dwindles, and then they introduce the ‘next big thing’. The companies are probably quite ready to introduce a whole slew of features, but are happy to doll them out year after year as ‘upgrades’, so long as, year after year, we buy them.
I have the Canon 40D. It has both a DOF preview and live view.
The DOF preview button allows you to stop down to the set aperture and see the actual DOF in the viewfinder. Too dark? Too small?
Then combine it with the liveview mode (combined with show true exposure for live view). When i push the DOF preview button in live view, it will calculate exposure for the given settings – you can see both the exposure and the DOF, live.
My point is that whatever photographers can think up that they want in their cameras, you can bet the camera companies have thought of it as well. They are just bound by marketing – they will sell whatever sells now, until it dwindles, and then they introduce the ‘next big thing’. The companies are probably quite ready to introduce a whole slew of features, but are happy to doll them out year after year as ‘upgrades’, so long as, year after year, we buy them.
I don’t want to get off topic here but as far as I know (being a former N95 owner) none of the Nokia software does not work on a Mac.
I don’t want to get off topic here but as far as I know (being a former N95 owner) none of the Nokia software does not work on a Mac.
Typo above… supposed to say “None of the nokia software works on a Mac”
Typo above… supposed to say “None of the nokia software works on a Mac”
It was about November 1999. And here’s some more details from :
“One of the most intriguing aspects of the DC290 is its incorporation of FlashPoint’s “Digita” scripting system, which allows you to write “scripts” (simple programs) to automate processes, or give the camera unique capabilities. The DC290 comes with several default scripts already installed, adding various functions. For example, the bracketing script shoots three images consecutively with three different EV values (you get to set the exposure increments). Another example is the Resolution Series script which shoots a range of pictures with different resolutions.”
It was about November 1999. And here’s some more details from :
“One of the most intriguing aspects of the DC290 is its incorporation of FlashPoint’s “Digita” scripting system, which allows you to write “scripts” (simple programs) to automate processes, or give the camera unique capabilities. The DC290 comes with several default scripts already installed, adding various functions. For example, the bracketing script shoots three images consecutively with three different EV values (you get to set the exposure increments). Another example is the Resolution Series script which shoots a range of pictures with different resolutions.”
The website I quoted was http://www.imaging-resource.com/PRODS/DC290/DC290A.HTM
The website I quoted was http://www.imaging-resource.com/PRODS/DC290/DC290A.HTM
Interesting. Thanks for that!
Interesting. Thanks for that!
Great thoughts. A few years ago, Volvo Cars played with the idea of letting owners upgrade their car’s horsepower via a web site or cell phone. Punching in a code would release more of the available (but software constrained) power. So, translated to your post, future cameras could be upgraded on-the-fly through software to meet sensor size demands for glass, light, motif and budget.
Great thoughts. A few years ago, Volvo Cars played with the idea of letting owners upgrade their car’s horsepower via a web site or cell phone. Punching in a code would release more of the available (but software constrained) power. So, translated to your post, future cameras could be upgraded on-the-fly through software to meet sensor size demands for glass, light, motif and budget.
I have also heard of the CHDK. The only question I have is how to convert the raw files. If it shoots in its own format then the converters will have to be updated. I can’t see the lightroom team(for example) unraveling the code for the five people that shoot with hacked cameras.
I have also heard of the CHDK. The only question I have is how to convert the raw files. If it shoots in its own format then the converters will have to be updated. I can’t see the lightroom team(for example) unraveling the code for the five people that shoot with hacked cameras.
Imagine this for image stabilization:
Instead of the camera shooting one exposure for say 1/4 of a second, the camera makes 20 exposures at 1/80 of sec and adds up all the information. You then get the exposure that looks like 1/4 of a sec but with the speed of a faster shutter. I’m just throwing these numbers out there as examples, but the algorithms are already around for aligning multiple photos. So we use that technology to make sharp images using multiple fast shutter speeds, or exposures.
–JZ
Imagine this for image stabilization:
Instead of the camera shooting one exposure for say 1/4 of a second, the camera makes 20 exposures at 1/80 of sec and adds up all the information. You then get the exposure that looks like 1/4 of a sec but with the speed of a faster shutter. I’m just throwing these numbers out there as examples, but the algorithms are already around for aligning multiple photos. So we use that technology to make sharp images using multiple fast shutter speeds, or exposures.
–JZ
Good idea. I know I would want one. Plenty of great thoughts.
Good idea. I know I would want one. Plenty of great thoughts.