software:unpaper_test
Differences
This shows you the differences between two versions of the page.
| Both sides previous revisionPrevious revisionNext revision | Previous revision | ||
| software:unpaper_test [2009/03/11 05:16] – admin | software:unpaper_test [2015/04/22 21:51] (current) – [findblack] admin | ||
|---|---|---|---|
| Line 1: | Line 1: | ||
| - | ====== Unpaper test ===== | + | ====== Unpaper test ====== |
| In order to better understand the vast amount of options in unpaper, I've done several tests, which show differences between settings.\\ I scanned a lot of sheetmusic, processed them manually with another application to remove skew. All actions have been done in 8-bit gray at a resolution of 300 dpi.\\ This test should show which settings should be done with unpaper to get a decent automatic conversion to black and white with despeckle and noise removal. | In order to better understand the vast amount of options in unpaper, I've done several tests, which show differences between settings.\\ I scanned a lot of sheetmusic, processed them manually with another application to remove skew. All actions have been done in 8-bit gray at a resolution of 300 dpi.\\ This test should show which settings should be done with unpaper to get a decent automatic conversion to black and white with despeckle and noise removal. | ||
| Line 34: | Line 34: | ||
| str1=" | str1=" | ||
| - | unpaper -b $str1 $donot -t pbm /tmp/$ptmpf-1.pgm | + | unpaper -b $str1 $donot -t pbm $ptmpf-1.pgm $ptmpf.pbm |
| # convert inputfile option annotation outputfile | # convert inputfile option annotation outputfile | ||
| - | convert | + | convert $ptmpf.pbm -gravity |
| - | convert -monochrome -density 300 -units PixelsPerInch | + | convert -monochrome -density 300 -units PixelsPerInch / |
| - | rm /tmp/$ptmpf.pbm | + | rm $ptmpf.pbm $ptmpf_ca.pbm |
| - | pdflst=" | + | pdflst=" |
| nmb=$(($nmb+1)) | nmb=$(($nmb+1)) | ||
| done | done | ||
| Line 83: | Line 83: | ||
| ===== Scripting with unpaper ===== | ===== Scripting with unpaper ===== | ||
| + | ==== gray2black ==== | ||
| Automatic conversion from pdf files, containing gray images, to black and white images with unpaper and its noise reduction and cleaning up features can be done with the following script: | Automatic conversion from pdf files, containing gray images, to black and white images with unpaper and its noise reduction and cleaning up features can be done with the following script: | ||
| <code bash> | <code bash> | ||
| #!/bin/sh | #!/bin/sh | ||
| - | # filename: | + | # filename: |
| - | # This script | + | # This script |
| + | # unpaper, imagemagick and ghostscript to a black and white pdf file. | ||
| + | # | ||
| + | # Usage: gray2black input-file [option] output-file | ||
| # | # | ||
| - | # usage: findblack.sh < | + | # option: -b [value] specify black threshold value as being used in |
| + | # | ||
| + | # be used: -b 0.12 | ||
| + | # input-file: a pdf file containing sheetmusic with one or more gray | ||
| + | # | ||
| + | # output-file: | ||
| + | # settings will be created. | ||
| # | # | ||
| - | # < | + | # Example: |
| - | # < | + | # Process gray images in mozart.pdf: gray2black mozart.pdf -b 0.23 test.pdf |
| + | # | ||
| + | # This script uses imagemagick to convert and center an image on an | ||
| + | # a4 page. (With 2480x3508 pixels at a resolution of 300 dpi) and | ||
| + | # ghostscript to create a small pdf file. By using efficiently the | ||
| + | # piping method (|) in conjunction | ||
| + | # standard input (-) we get less tempfiles. | ||
| # | # | ||
| - | # Observe the outputfile.pdf and find the page which has the best value for black-threshold. (Text is embedded) | + | # NB. This script is time consuming. So have a lot of patience. |
| + | # Each page will take about 15 seconds to execute on a single core | ||
| + | # AMD64 3500+ cpu. | ||
| # | # | ||
| - | # | + | # User text |
| - | # Use imagemagick to convert and center an image on an a4 page. (With 2480x3508 pixels at a resolution of 300 dpi). | + | _fbhelp=" |
| - | # Use ghostscript to create a small pdf file. | + | _fbspdf=" |
| - | # reading and writing to the standard input (-) we get: | + | _fbcvrt=" |
| + | # Unpaper settings, start value for black-threshold is 0.12 | ||
| + | b_threshold=" | ||
| + | donot=" | ||
| + | filter=" | ||
| + | # Other settings | ||
| + | ptmpf=" | ||
| + | |||
| + | # Page dimensions | ||
| psize=" | psize=" | ||
| - | pdim="2480x3508" | + | horpix=2480 |
| + | verpix=3508 | ||
| + | pdim="$horpix" | ||
| pres=" | pres=" | ||
| - | convert -size $pdim xc: | + | |
| - | # This will take about 5 seconds for a single page to execute | + | # Only resize if absolutely neccessary. Values for resizing are for left, right, top and bottom border. |
| + | hborder=33 | ||
| + | vborder=33 | ||
| + | lhorpix=$horpix | ||
| + | lverpix=$verpix | ||
| + | |||
| + | # Store input-file name temporarily, | ||
| + | arg1=$1 | ||
| + | |||
| + | # Find number of parameters passed | ||
| + | if [ " | ||
| + | echo " | ||
| + | exit 1 | ||
| + | fi | ||
| + | |||
| + | # Check OPTIONS | ||
| + | if [ " | ||
| + | case " | ||
| + | * ) | ||
| + | echo " | ||
| + | exit 1 | ||
| + | ;; | ||
| + | esac | ||
| + | elif [ " | ||
| + | case " | ||
| + | -b ) | ||
| + | shift | ||
| + | b_threshold=$2 | ||
| + | shift | ||
| + | ;; | ||
| + | * ) | ||
| + | echo " | ||
| + | exit 1 | ||
| + | ;; | ||
| + | esac | ||
| + | fi | ||
| + | |||
| + | # only process further if there are two arguments left. | ||
| + | if [ " | ||
| + | if [ ! " | ||
| + | echo " | ||
| + | exit 1 | ||
| + | else | ||
| + | # convert first page to a pgm file with filename $ptmpf-*.pgm | ||
| + | pdftoppm -gray -r 300 " | ||
| + | |||
| + | # Check for document resize neccessity. Iterating through all pages, makes sure, all pages get uniform resizing later. | ||
| + | |||
| + | resize=0 | ||
| + | # Iterate through files. We need to sort files in a numerical order because pdftoppm generates files without a leading zero. | ||
| + | # However before the numbers, a ' | ||
| + | for filenum in $(ls $ptmpf-*.pgm | cut -d' | ||
| + | dimension=" | ||
| + | # width: " | ||
| + | # height: " | ||
| + | if [ ${dimension%%x*} -gt $lhorpix ]; then | ||
| + | lhorpix=${dimension%%x*} | ||
| + | resize=1 | ||
| + | fi | ||
| + | if [ ${dimension## | ||
| + | lverpix=${dimension## | ||
| + | resize=1 | ||
| + | fi | ||
| + | done | ||
| + | # Resize document if neccessary | ||
| + | if [ $resize -eq 1 ]; then | ||
| + | # image fits within horizontal boundary, so adaptation must be vertical | ||
| + | if [ $horpix -eq $lhorpix ]; then | ||
| + | vscale=$(( ($verpix-30)*100/ | ||
| + | else | ||
| + | vscale=100 | ||
| + | fi | ||
| + | # image fits within vertical boundary, so adaptation must be horizontal | ||
| + | if [ $verpix -eq $lverpix ]; then | ||
| + | hscale=$(( ($horpix-30)*100/ | ||
| + | else | ||
| + | hscale=100 | ||
| + | fi | ||
| + | # scaling neccessary both horizontal and vertical | ||
| + | if [ $hscale -lt $vscale ]; then scale=$hscale; | ||
| + | |||
| + | echo Some images exceed the maximum size. Now scaling to " | ||
| + | # iterate through all files and resize all with the same percentage, use 8 bits per pixel | ||
| + | for filenum in $(ls $ptmpf-*.pgm | cut -d' | ||
| + | mogrify -depth 8 -resize " | ||
| + | echo resizing file: " | ||
| + | done | ||
| + | fi | ||
| + | # apply unpaper onto each page | ||
| + | for filenum in $(ls $ptmpf-*.pgm | cut -d' | ||
| + | # Parameter expansion: ${param%word} From the end remov? Mis je nog iets?es the smallest part of param that matches word and returns the rest. (p.72) | ||
| + | unpaper -b $b_threshold $filter $donot -t pbm " | ||
| + | rm " | ||
| + | done | ||
| + | |||
| + | # center pbm page on an a4 canvas, convert to pdf | ||
| + | pdflst="" | ||
| + | for filenum in $(ls $ptmpf-*.pbm | cut -d' | ||
| + | convert -size $pdim xc: | ||
| + | rm " | ||
| + | pdflst=" | ||
| + | done | ||
| + | |||
| + | # merge all pages into on page | ||
| + | gs -sDEVICE=pdfwrite -dNOPAUSE -dQUIET -dBATCH " | ||
| + | rm $pdflst | ||
| + | fi | ||
| + | fi | ||
| + | exit 0 | ||
| </ | </ | ||
| - | Find the value for black threshold with following script: | + | ==== findblack ==== |
| + | Find a value for black threshold with following script: | ||
| <code bash> | <code bash> | ||
| #!/bin/sh | #!/bin/sh | ||
| - | # filename: findblack.sh | + | # filename: findblack |
| # This script helps to find an unpaper' | # This script helps to find an unpaper' | ||
| # creating a series of pdf files each with a different black threshold setting. By observing | # creating a series of pdf files each with a different black threshold setting. By observing | ||
| Line 116: | Line 253: | ||
| # annotated on top of each page. | # annotated on top of each page. | ||
| # | # | ||
| - | # usage: findblack.sh [optional: pagenumber] < | + | # usage: findblack |
| # | # | ||
| - | # pagenumber: which page should be used as source for unpaper. If not specified, first page will be taken. | + | # option: -p [number] |
| - | # < | + | # input-file: |
| - | # < | + | # output-file: |
| # | # | ||
| + | # Example: | ||
| + | # Process page 4 from mozart.pdf: ./findblack mozart.pdf -p 4 test.pdf | ||
| # | # | ||
| # This script uses imagemagick to convert and center an image on an a4 page. (With 2480x3508 pixels at a resolution of 300 dpi) | # This script uses imagemagick to convert and center an image on an a4 page. (With 2480x3508 pixels at a resolution of 300 dpi) | ||
| Line 128: | Line 267: | ||
| # | # | ||
| # NB. This script is time consuming. So have a lot of patience. | # NB. This script is time consuming. So have a lot of patience. | ||
| - | # The line: " | + | # Each page will take about 15 seconds to execute on a single core AMD64 3500+ cpu. |
| - | # 10 seconds | + | |
| - | # Written on march 2009, m.nijdam | + | |
| + | # User text | ||
| + | _fbhelp=" | ||
| + | _fbspdf=" | ||
| + | _fbcvrt=" | ||
| - | # start value for black-threshold is 0.05 | + | # Unpaper settings, |
| a=5 | a=5 | ||
| - | # end value for black-threshold is 0.40 | + | b=20 |
| - | b=40 | + | s=5 |
| donot=" | donot=" | ||
| + | # Other settings; $$ references the current PID | ||
| ptmpf=" | ptmpf=" | ||
| psize=" | psize=" | ||
| Line 146: | Line 287: | ||
| ptxt=" | ptxt=" | ||
| pstr=" | pstr=" | ||
| + | # Default page number | ||
| + | defpage=1 | ||
| # Find number of parameters passed | # Find number of parameters passed | ||
| if [ " | if [ " | ||
| - | echo "Need input filename and output filename for pdf file" | + | echo "$_fbhelp" |
| exit 1 | exit 1 | ||
| fi | fi | ||
| - | ptst=`identify | + | # Store input-file name temporarily, |
| - | if [ ! "$ptst" ]; then | + | arg1=$1 |
| - | echo "please supply a pdf file" | + | |
| + | # Check OPTIONS | ||
| + | if [ " | ||
| + | case " | ||
| + | * ) | ||
| + | echo " | ||
| + | exit 1 | ||
| + | ;; | ||
| + | esac | ||
| + | elif [ " | ||
| + | case " | ||
| + | -p ) | ||
| + | shift | ||
| + | defpage=$2 | ||
| + | shift | ||
| + | ;; | ||
| + | * ) | ||
| + | echo " | ||
| + | exit 1 | ||
| + | ;; | ||
| + | esac | ||
| + | fi | ||
| + | |||
| + | # only process further if there are two arguments left. | ||
| + | if [ " | ||
| + | if [ ! "$(identify " | ||
| + | echo "$_fbspdf" | ||
| + | exit 1 | ||
| else | else | ||
| + | # find last page, start counting from 0, necessary for proper deleting pdftoppm result file which may have extra 0 | ||
| + | trail="" | ||
| + | lastPage=" | ||
| + | if [ " | ||
| + | trail=" | ||
| + | elif [ " | ||
| + | trail=" | ||
| + | elif [ " | ||
| + | trail=" | ||
| + | elif [ " | ||
| + | trail=" | ||
| + | fi | ||
| # convert first page to a pgm file with filename tmpfile_PID-1.pgm | # convert first page to a pgm file with filename tmpfile_PID-1.pgm | ||
| - | pdftoppm -gray -r 300 -f 1 -l 1 $1 /tmp/$ptmpf | + | pdftoppm -gray -r 300 -f $defpage |
| # check for succesful conversion to .pgm | # check for succesful conversion to .pgm | ||
| - | if [ ! -f "/tmp/$ptmpf-1.pgm" ]; then | + | if [ ! -f " |
| - | echo "Can't find converted first page of pdf file. Check your pdf file" | + | echo "$_fbcvrt" |
| - | rm "/tmp/$ptmpf-1.pgm" | + | rm " |
| exit 1 | exit 1 | ||
| else | else | ||
| pdflst="" | pdflst="" | ||
| + | |||
| nmb=$a | nmb=$a | ||
| while [ " | while [ " | ||
| Line 176: | Line 358: | ||
| str1=" | str1=" | ||
| fi | fi | ||
| + | |||
| # process unpaper | # process unpaper | ||
| - | unpaper -b $str1 $donot -t pbm /tmp/$ptmpf-1.pgm /tmp/$ptmpf.pbm | + | unpaper -b $str1 $donot -t pbm $ptmpf-$trail$defpage.pgm $ptmpf.pbm |
| + | |||
| # Create white A4 size canvas -> center pbm file into canvas -> convert pbm to ps -> convert ps to pdf | # Create white A4 size canvas -> center pbm file into canvas -> convert pbm to ps -> convert ps to pdf | ||
| # New to bash? | # New to bash? | ||
| Line 185: | Line 367: | ||
| # Sending result to next command with pipe: | | # Sending result to next command with pipe: | | ||
| # Using this ' | # Using this ' | ||
| - | convert -size $pdim xc:white miff:- | composite -density $pres -units PixelsPerInch -compose atop -gravity Center | + | convert -size $pdim xc:white miff:- | composite -density $pres -units PixelsPerInch -compose atop -gravity Center $ptmpf.pbm - miff:- | convert - $pstr "$ptxt $str1" - | convert -monochrome -density 300 -units PixelsPerInch - ps:- | ps2pdf13 -sPAPERSIZE=a4 - " |
| - | + | ||
| - | pdflst=" | + | pdflst=" |
| - | nmb=$(($nmb+1)) | + | nmb=$(($nmb+$s)) |
| - | rm | + | rm $ptmpf.pbm |
| done | done | ||
| - | rm /tmp/$ptmpf-1.pgm | + | rm $ptmpf-$trail$defpage.pgm |
| # Merge all pdf files in $pdflst into one single file with filename $2 | # Merge all pdf files in $pdflst into one single file with filename $2 | ||
| gs -sDEVICE=pdfwrite -dNOPAUSE -dQUIET -dBATCH -sOutputFile=$2 $pdflst | gs -sDEVICE=pdfwrite -dNOPAUSE -dQUIET -dBATCH -sOutputFile=$2 $pdflst | ||
| Line 197: | Line 379: | ||
| fi | fi | ||
| fi | fi | ||
| + | fi | ||
| exit 0 | exit 0 | ||
| </ | </ | ||
| + | |||
| + | ===== Bugs ===== | ||
| + | [[software: | ||
software/unpaper_test.1236745005.txt.gz · Last modified: 2009/03/11 05:16 by admin