logo
background
 Home and Links
 Your PC and Security
 Server NAS
 Wargames
 Astronomy
 PhotoStory
 DVD making
 Raspberry Pi
 PIC projects
 Other projects
 Next >>

Using the Pi Camera

Pi Camera usage
You can find the latest raspistill etc. parameter definitions on github as an ODT document RaspiCamDocs.odt. For those unwilling to download Open Office, here's a simple text snapshot as of 2016-08-19

Image capture (with the 5M pixel Pi-camera)

First you have to 'enable' the camera via 'raspi-config'. To change the settings enter (sudo raspi-config). In the options window, tab or mouse cursor down to the 'Enable Camera' option, press 'return' and select 'yes' .. then exit and reboot

Note - the camera software will give you a 'hint' if you try to take a photo without using raspi-config to enable the camera.
 
If it still won't work after you 'enable', it's time to look for a loose cable etc.

To capture single images use raspistill (or, for RAW, raspiyuv). To capture video use raspivid / raspividyuv (or v4l2 (Video for Linux v2) / UV4L (yuv video 4 linux)). Some documentation for the camera can be found at www.raspberrypi.org/documentation/raspbian/applications/camera.md and www.raspberrypi.org/documentation/hardware/camera.md. For the most up=-to-=date command options, type 'raspistill' at the command prompt

The '-md' option sets the 'mode'. Camera modes are :-

-md 0	"automatic" selection (based on requested frame rate)
-md 1 	1920x1080 	16:9 	1-30fps HD (default video)
-md 2 	2592x1944 	4:3 	1-15fps full frame (default stills)
-md 3 	2592x1944 	4:3 	0.1666-1fps
-md 4 	1296x972 	4:3 	1-42fps (2x2 binning)
-md 5 	1296x730 	16:9 	1-49fps (2x2 binning)
-md 6 	640x480 	4:3 	42.1-60fps 	(2x2 binning plus skip)
-md 7 	640x480 	4:3 	60.1-90fps 	(2x2 binning plus skip)
 
The fps limits apply to movie mode (not stills = still images are limited by the sensor read out time, which takes about 200mS for the whole array)
 
The HD modes all (centre aligned) 'crop' the sensor. 'Binning' combines a 4x4 pixel array into a single pixel (thus increasing low-light sensitivity).
 
Speed is limited by how much of the camera has to be read out (by the GPU).

Note that '15 fps full frame' is only possible in VIDEO MODE (h264), not in 'still image' (JPEG) mode. In still image mode, it takes 200mS to 'read' the sensor (this is on top of the 'exposure' time) which gives us a 'theoretical' 5 fps (assuming '0' exposure time, say in bright sunlight, when 1/1000th = 1mS might be realistic). So data output rates for video are about 10x faster than that for still images. 15 fps video uses about the same bandwidth as 1.5 fps jpeg.

With the recent release of the Pi camera 2, you can expect rapid changes. So it's a 'good idea' to start by updating your Pi system software  :-
 
sudo apt-get update && sudo apt-get upgrade -y
 
and GPU firmware :-
sudo rpi-update
 
Then reboot
sudo reboot

Capturing video with raspivid

The default video capture generates a clip 5 seconds long (note horizontal and vertical flip options are same as raspistill)

raspivid -o file-name.h264

To set the video length, add -t length in mS option, for example, for a 30 second video :-

raspivid -o file-name.h264 -t 30000

To view the video on the Pi, you can use omxplayer :-

omxplayer -o hdmi file-name.h264

Capturing still photographs with raspistill / raspiyuv

Note that YUV (RAW) is the camera native format, so this comes out 'first', whilst JPEG (RGB) - which is GPU hardware accelerated - takes 'a little longer' (there may be only 20mS or so difference but, when trying to get the max. still photo speed (5 fps) it can make a difference

On 'start up', the camera is in 'preview' mode and will take some frames to 'auto-white balance' and 'work up' it's 'auto-grain-control' (AGC).
 
Turning off AGC actually 'freezes' it at the current level (and it starts at zero), so unless you give the camera time (500mS or so) to 'balance' and get it's AGC right, you will (usually) end up with completely underexposed (black) frames (this is especially a problem for Python users who 'go direct' to the camera).
 
The built-in utilities,  raspistill / raspiyuv and raspivid / raspividyuv, all default to the correct 'pre-amble' (on the still photo side, this is why you 'always' loose the second frame of a burst = see later)

The default (full) resolution is 2592 x 1944 pixels and a typical .jpg image size will be about 2.2Mb (allow 2.5Mb to be safe). To take a photo and output (-o) it to a file eg. file-name.jpg you can use the commands:-

raspiyuv -o file-name.raw
This runs the camera for 5 seconds and then captures a still at max. resolution (2592x1944). The RAW file is 2592x1952 'pixels' and the YUV format is 'YUV420p' (which is 6 bytes per 4 pixels) giving 7.5 Mb per frame.
 
raspistill -o file-name.jpg
This runs the camera for 5 seconds and then captures a still at max. resolution (2592x1944). The data is coveted from YUV into RGB and JPEG compressed by the GPU. The JPEG file is 2592x1944 pixels and typically about of 2.2Mb

For more on the camera RAW data, see Wikipedia

The '--colfx' flag (-cfx) allows you to modify the U and V data 'depth'. Setting '-cfx 128:128' shifts all the information into the  Y channel (i.e. generates a monochrome image).

For more on the camera modes, see Raspberry Pi documentation

Setting the total capture time (-t) and the single photo time (-tl)

The -t parameter sets the 'total capture time' in mS i.e. the length of a video clip. When capturing single images, -t sets the 'start delay'. When taking multiple images -t sets the time for the whole 'batch' and the -tl time (in mS) is the 'gap' between images.

If not specified, -t is 5 seconds (i.e. same as -t 5000)
-t 0 means 'run for ever' ?? (Ctrl+c aborts)

The -tl is the time lapse between shots, in milliseconds

If a time-lapse value of 0 is entered, the application will take pictures 'as fast as possible'.
 
Note there is an minimum enforced pause of 30ms between captures to ensure that exposure calculations can be made.

Maximum photos per second

To maximise the still photo rate, keep the camera in 'burst mode', set the fastest 'output format' (this is jpeg+no thumbnail = see below) and minimise the photo file destination write time (RAM-disk is quickest, then Class 10 SDHC, o/p to USB stick usually next then Ethernet). So, first we have to 'set up' for max. speed :-



(-) High speed photos


Max. photos per second

In still image (photo) mode, all pixels have to be read and this takes 200mS. So 'in theory' we can achieve 5 fps (frames per second) - but what's the actual maximium ?

Preventing Preview

The no preview flag (-n) doesn't actually 'stop' previews - instead it just sends the preview data to DEV NUL. However this does 'save some time' (since normally the data would have to be directed to the display memory).

A Previews is a short video sequence. For videos, something called 'rolling shutter mode' is used, whilst for still photo capture the sensor is operated in a different mode.

This means the camera has to be switched from Preview (video) mode to still mode before a photo can be taken - and this not only takes time but also corrupts the first 'frame' read in still capture mode. Even worse, at the then of each photo 'by default' the camera drive switches back to preview mode !

To avoid all the time (and 'first frames') lost switching modes, a new flag was intruduced called 'burst mode'. When the -bm flag is used, the camera does not re-enter Preview after the photo has been taken.

Note that the camera has to start in Preview mode because that's when it's setting is Automatic Gain Control (AGC). The AGC value starts at zero and is 'wound up' until image data is found (0 means 'totally underxposed' i.e. black frame). It's possible to 'stop' the AGC (which freezes it at whatever value it's reached) however that's usually 'not a good idea' :-)


Setting the -tl time

You might think that setting -tl to 200 (200 mS = 5 fps) would get you the maximium possible number of frames. Well 'yes', but if the rest of the system 'can't keep up' (convert to jpg, write to SDHC etc) you will end up reading frames 'late' ... and once the first is late the problem is likley to 'cascade' until you end up skipping frames.

So, for max. speed, you set -tl to 0 (which means the camera runs for the -t time generating photos as fast as the rest of the system asks for)

JPEG or YUV ?

The first thing 'output' from the camera is YUV - this then has to be processed into whatever final output format you ask for. Some formats - such as jpeg - are processed by the GPU, so take 'almost no' extra time.

Even so, YUV will be avialble faster than jpeg. The 'problem' with choosing YUV is that the image size will be 7.5Mb - and it's slower to strore 7.5Mb than it is to convert to jpeg and stor the (typical) 2.2 Mb result.

Dropping the thumbnail

By default, a thumbnail (-th w:h:q) is added to jpeg photos. This takes time (and uses extra space), so setting '--thumb none' (or '-th 0:0:0') speeds up the jpeg photo output.

Writing to RAM disk

There does not appear to be much difference between writing to a good Class 10 SDHC card or writing to 'tmpfs' RAM disk - however, ever little bit counts

This note last modified: 17th Nov 2017 17:30.

[top]


The command to generate 10 seconds of max. frame rate jpeg photos to SDHC is :-   sudo raspistill -n -bm -o %04d.jpg -t 10000 -tl 0 --thumb none   The '%04d' means 'generate a 4 digit ascending serial number' for the file name (note that when a frame is 'skipped', because time runs out, so is it's serial number). For more on auto-file naming, see :-

(+) Auto file names


To annotate the photo with an actual time-stamp (which is only accurate if the Pi has access to a Time Server via a live Internet connection, else it's down to the internal clock), see :-

(+) Image anotation


To set-up a RAM-disk (to maximise photo save speeds), see :-

(+) Pi ram disk - (tmpfs)

At maximum resolution ()

My results Max read out rate from a Pi B+, NoIR 5Mpixel camera, generic Class 10 SDHC, arm_freq=900, core_freq=333, sdram_freq=450. The camera is indoors and fixed pointing at a bookshelf, so all images 'should' be identical

sudo raspistill -n -bm -o %02d.jpg -t 10000 -tl 0 --thumb none
 
Successive tests (saving to a Class 10 SDHC) got me 28, 20, 18, 24, 21 ..
 
Not only was this very disappointing, it was also very inconsistent. What's more, despite not moving the camera, the file sizes varied from 'batch to batch' eg. 2,631-2,640kb, 2,596-2,599kb, 2,590-2,595kb, 2,590-2,656kb ??, 2,604-2,608kb. Note that in general images varied by less then 10kb within a batch - except when the variation was 66kb ??
 
So I tried again, this time saving to 'ram-disk' (tmpfs) :-
 
Successive tests got me 38, 38, 36, 31, 35. Again each the 'batch' of images varied from 2,449-2,460kb to 2,618-2,625kb. Images within a batch varied by up to 10kb or so.
 
Whilst the max. photo frames per second (fps) is constrained by the 'save' speed, 'something else' is affecting achieved fps
 
Guessing that the variation was down to 'auto expose' decisions, I forced the ISO to 800 :-
sudo raspistill -ISO 800 -n -bm -o %02d.jpg -t 10000 -tl 0 --thumb none
 
Now I get 35 photos (almost) every time (to ram-disk). The file size has increased but there is less variation from batch to batch 3,208-3,228kb to 3,225-3,242kb (with one 'odd' 32 photo batch 3,509-3,531kb batch)

At lower resolutions

I noted that the 'odd' batch with the larger file sizes had been generated from a smaller count. So, to get the count up, I decided to drop the output size

Testing at 640x480 (and saving to ram-disk) :-
sudo raspistill -ISO 800 -n -bm -o %02d.jpg -t 10000 -tl 0 --thumb none -w 640 -h 480
 
This gets me 50 images, all of 225kb !
 
Exploring sizes, 1024x1280 still gets me 50 photos, size 999-1001kb.
 
1400x1050 continues the 50 photo breakthrough, with file sizes now 1125-1129kb
 
Finally :-
sudo raspistill -ISO 800 -n -bm -o %02d.jpg -t 10000 -tl 0 --thumb none -w 1600 -h 1200
Got me 50 photos at 1600x1200 (to ram-disk)  with the files now ranging from 1420-1430kb.
 
Trying the same whilst saving to SDHC reduced the image count to 37, with file sizes from 1399-1405kb.

There seems little doubt that the main limit to maximum fps is 'how fast' the images can be saved.

This suggests that the Pi Zero - which operates in 'turbo mode' by default - should achieve higher photo fps rates 'out of the box'

Upping the SoC frequencies

Increasing the camera read-out speed and ram frequencies should result in higher fps when saving full sensor frames to the ram-disk. For more on over-clocking see :-

(+) Pi Overclocking


I modified the /boot/config.txt as follows :- arm_freq=800 # from default 700 gpu_freq=350 # from 250 avoid_pwm_p11=1 # decouple core core_freq=450 # includes camera pipeline - up from 250 sdram_freq=450 # up from 400

Testing with full size saves to ramdisk :-

sudo raspistill -ISO 800 -n -bm -o %02d.jpg -t 10000 -tl 0 --thumb none
This gets me 39-40 photos (multiple tests), 4 more than previous, so upping the frequencies does seem to work

Working with thumbnails

Whilst adding a thumbnail increases the file size and drops the fps by 3% or so, adding a low res thumbnail to the photo at 'capture' time is faster than processing the full photo later.

The advantage of using thumbnails for things like 'motion detect' is that the image processing times are significantly reduced

To extract the thumbnail from the .jpg, install exiv2 and use the command exiv2 -et img0001.jpg (to extract img0001-thumb.jpg from img0001.jpg)

sudo apt-get install exiv2 
To extract all thumbs from jpegs in the current folder :- exiv2 -et *.jpg The default thumbnail are 24kb ...

Once you have the thumbnail it's possible to perform 'motion detection' at much higher speeds than using the full resolution photos My rather crude method is to compare the file sizes. If they are equal, I assume no motion and drop the older image. If they differ by more than a few bytes, the old image is kept as the reference whist the new is saved. I continue to save until the new is the same as the old reference, at which point that new becomes the reference.

Adding RAW

You can add the RAW data to the JPEG with the -r option.

As you might expect this increased the photo file size to approx 8.5Mb and killed the frame rate
 
Destination Class 10 SDHC, 0.9 fps
Destination ram-disk, 2 fps

I 'cd ..' until I reached /my-folder-for-PC' and then ran:- sudo raspistill -n -bm -o (ram-disk)/%04d.jpg -t 10000 -tl 200. This delivered the same photos count (39) to both SDHC and ram-disk in 10s (so 3.9 fps), however the command window filled with 'Frame X is nn ms late' complaints.

The best I managed was by adjusting the -tl to 220, which got me 45 photos to ram-disk, with almost every photo generating a 'Frame X is nn ms late' complaint.
 
The maximum 'repeatable' speed to ram-disk (with no 'late' complaints and the usual 'skipping 2 to restart at 3') was -tl 250, for 40 photos (4 fps)
 
The maximum 'repeatable' speed to ram-disk (with no 'late' complaints and only 3 'skipping X to restart at N') was -tl 270, for 30 photos (3 fps)

Conclusion = over-clocking makes no difference to the achievable RAW photo fps

Viewing Pi camera images

On the Pi itself

To view an image use 'fbi' :-

get and install fbi :-
sudo apt-get install fbi
 
If you are logged in locally (and are using the HDMI display), you can display an image with :-
fbi -d /dev/fb0 -a -noverbose img-file-name.jpg
 
If you are controlling the Pi from a remote ssh putty terminal window on the PC, use :-
sudo fbi -T 1 -a -noverbose img-file-name.jpg
 
-T 1 means use the video (HDMI, or TV out (RCA) if that is enabled instead) display, not the (text) terminal window :-), -o means resize to fit the screen and -noverbose means don't add text overlay
If you get the "ioctl VT_GETSTATE: Invalid argument (not a linux console?)" error, try '-T 2' instead of '-T 1'

On your PC

Any photo app will view the jpg images without problems. Video, however, is another matter.

Pi video is a 'transport stream' (.ts file type) i.e. an 'open ended' data source which can be 'streamed' across the Ethernet to your PC. About the only app. that can cope with this sort of 'live feed' on the PC is VLC

Pi-camera app. option flags

As usual, the Documentation ('-man') lags the functionality. Each new version of raspistill etc. likely has some new functions. These will often be in response to requests and issues highlighted on the Forums, so it's advisable to 'join in' the discussions :-)

For a full list of option flags, type raspistill (or raspiyuv etc) at the command prompt without any flags

Photo shutter speed and ISO

The '-ss n' option sets the shutter speed (in microseconds). The max. limit is approximately 6000000 uS (6 seconds). If not specified, 'auto' is used

The '-ISO n' option sets the 'sensitivity'. '-ISO 6400' resulted in 640 whilst '-ISO 999' resulted in 800 (which appears to be the maximum). If not specified, 'auto' is used (which seems to be ISO 100 most of the time)

Unless you can control the illumination, i.e. the camera is in-doors, then you have little choice but to allow auto since changing sun position and cloud cover would otherwise result in under or over exposure and even more 'motion' diffferences between shots.

Photo orientation

To rotate 180, use the horizontal and vertical flip option flags :-

raspistill -vf -hf -o file-name.jpg
This effects how the GPU reads out the data, so the impact on frame rate is, apparently, minimal

Dynamic Range compression (-drc, default off)

DRC improves the performance of the camera in low light conditions by increasing the range of values used for dark areas of the image (and thus decreasing the range for brighter areas). This can improve the image in low light areas.

Settings are :-
-drc off default
-drc low
-drc medium
-drc high
 
This just effects how the GPU generates the jpeg so should have zero impact of frame rate.

Finding the newest image

To save time searching file names by date, you can ask the camera app. to create a 'link' for you

-l 
Unlinks  from any previous image frame and links it to the latest frame

Converting JPEG to MJPEG (or video)

The Pi camera has both a MJPEG direct mode and various video modes, all of which are much faster (at least 15fps) than the still image mode (where you are lucky to achieve 4 fps). So below only applies if you took a sequence of still images that you now need to pack together

You can do this on the Pi using 'mencoder', but since it's done using the CPU, it will likely be a lot faster to transfer all the image files to your PC instead :-)

If you must do it on the Pi, start by getting memcoder

sudo apt-get install mencoder

Generate a list of the images to be stitched (navigate to the folder containing all your images and list the file names in to a text file):-

ls *.jpg > stills.txt

Do the stitch :-

mencoder -nosound -ovc lavc -lavcopts vcodec=mpeg4:aspect=16/9:vbitrate=8000000 -vf scale=1920:1080 -o timelapse.avi -mf type=jpeg:fps=24 mf://@stills.txt

The mencoder parameters should be self-explanatory

Streaming video to your PC etc.

It's easy enough use the raspivid utility to generate a raw H.264 video stream, however getting this off the Pi 'in real time' (rather than as a series of completed small video clips) is another matter. There are a number of possible solutions

1) TCP/IP Streaming Using netcat

This is by far the simplest approach. All you do is 'pipe' ( using the | key, back-slash shifted, the key next to the left shift key on most keyboards) the output from raspivid to netcat (nc) which then sends the stream out over a chosen TCP/IP port. The command (to generate a half HD video stream) and send it out over port 5001, is:-


raspivid -n -ih -t 0 -rot 0 -w 1280 -h 720 -fps 15 -b 1000000 -o - | nc -lkv4 5001

The nc (netcat) parameter '-lkv4' breaks down as follows :-
-l (and 5001) = 'listen' (actually, transmit) on port 5001
k = constant connection
v = verbose error messages
4 = use the Pi's IPv4 address

The disadvantage of using netcat (nc, or for multiple destinations, ncat) is that you need software 'at the other end' to view the video = on the PC that's VLC, and on an android tablet you can use RPi Camera Viewer for Android

If you want to view the stream 'across the Internet' however, you need a HTTP streamer

2) HTTP Streaming Using UV4L

Whilst netcat gets raw streaming video onto the LAN it requires 'clever' software at the 'other end' to pick it up. Fortunately, we can use the well know "Userspace Video4Linux2 (v4l2)" driver for the Raspberry Pi CSI Camera to send to any standard web browsers using http.

In fact, UV4L can also be used to send steaming video to 'motion', 'MJPG-streamer', 'SimpleCV', 'fswebcam' and a host of other tools. For more information on installing see here. For more information on setting up with Wheezy, see this surveillance with Raspberry Pi NoIR camera page

To setup the UV4L driver for Jessie (using curl rather than wget) :-


curl http://www.linux-projects.org/listing/uv4l_repo/lrkey.asc | sudo apt-key add -
edit the  '/etc/apt/sources.list' file to add the line "deb http://www.linux-projects.org/listing/uv4l_repo/raspbian/ jessie main". You can do this by using the 'tee' utility as follows :-
echo "deb http://www.linux-projects.org/listing/uv4l_repo/raspbian/ jessie main" | sudo tee -a /etc/apt/sources.list
now update
sudo apt-get update
next I now recommend you install the various uv4l packages one at a time (many guides just 'concatenate' them all onto one line, which is fine SO LONG AS NOTHING GOES WRONG ..
sudo apt-get install -y uv4l uv4l-raspicam
sudo apt-get install uv4l-raspicam-extras
sudo apt-get install uv4l-server
sudo apt-get install uv4l-uvc
The next is required only if you intend to stream to the Pi's 'X' GUI
sudo apt-get install uv4l-xscreen
The next is required only if you intend to stream JPEG's (as well as h264 video)
sudo apt-get install uv4l-mjpegstream
If all installed OK, reboot
sudo reboot

To test, run the streaming video server at VGA resolutions and view the stream on your PC (by entering http://{P IP address}:8080 in your PC browser address bar). You should get a VGA video at about 30 fps.

:-

uv4l --driver raspicam --auto-video_nr --encoding h264 --width 640 --height 480 --enable-server on

To stop the video stream server :-

  

The UV4L parameters are stored in /etc/uv4l/uv4l-raspicam.conf. The following settings will get you the same video as the default raspivid command :-


sudo nano /etc/uv4l/uv4l-raspicam.conf
set the following for a default raspivid video :-
encoding = h264
width = 1280
height = 720
framerate = 15
bitrate = 1000000
inline-headers = yes
the default port is :8080, so it does no harm to add the following (which will remind you how to change it) :-
server-option = –port=8080
you can optionally set Browser user passwords :-
server-option = –admin-password=password
server-option = –config-password=password
 
After making changes, restart the uv4l_raspicam service (or reboot the Pi)
sudo service uv4l_raspicam restart
 
Set the capture format for video: $ v4l2-ctl --set-fmt-video=width=1920,height=1088,pixelformat=4 (Note: 1088 not 1080). Capture: $ v4l2-ctl --stream-mmap=3 --stream-count=100 --stream-to=somefile.h264 Stills capture: $ v4l2-ctl --set-fmt-video=width=2592,height=1944,pixelformat=3 $ v4l2-ctl --stream-mmap=3 --stream-count=1 --stream-to=somefile.jpg List of available formats: $ v4l2-ctl --list-formats

Next page :- Pi Camera iSpyServer

[top]