software:unpaper_test
Differences
This shows you the differences between two versions of the page.
| Both sides previous revisionPrevious revisionNext revision | Previous revision | ||
| software:unpaper_test [2009/03/11 00:21] – admin | software:unpaper_test [2015/04/22 21:51] (current) – [findblack] admin | ||
|---|---|---|---|
| Line 1: | Line 1: | ||
| - | ====== Unpaper test ===== | + | ====== Unpaper test ====== |
| In order to better understand the vast amount of options in unpaper, I've done several tests, which show differences between settings.\\ I scanned a lot of sheetmusic, processed them manually with another application to remove skew. All actions have been done in 8-bit gray at a resolution of 300 dpi.\\ This test should show which settings should be done with unpaper to get a decent automatic conversion to black and white with despeckle and noise removal. | In order to better understand the vast amount of options in unpaper, I've done several tests, which show differences between settings.\\ I scanned a lot of sheetmusic, processed them manually with another application to remove skew. All actions have been done in 8-bit gray at a resolution of 300 dpi.\\ This test should show which settings should be done with unpaper to get a decent automatic conversion to black and white with despeckle and noise removal. | ||
| Line 23: | Line 23: | ||
| <code bash> | <code bash> | ||
| #!/bin/sh | #!/bin/sh | ||
| - | donot=" | + | |
| + | donot=" | ||
| + | |||
| + | pdflst="" | ||
| + | ptmpf=" | ||
| nmb=0 | nmb=0 | ||
| while [ " | while [ " | ||
| do | do | ||
| - | str1=" | + | str1=" |
| - | unpaper $str1 | + | |
| - | # convert inputfile option annotation outputfile | + | unpaper -b $str1 $donot -t pbm $ptmpf-1.pgm $ptmpf.pbm |
| - | convert | + | |
| - | convert -density 300 -units PixelsPerInch | + | # convert inputfile option annotation outputfile |
| - | rm outc.pbm | + | convert |
| - | rm outca.pbm | + | convert |
| + | rm $ptmpf.pbm $ptmpf_ca.pbm | ||
| + | pdflst=" | ||
| nmb=$(($nmb+1)) | nmb=$(($nmb+1)) | ||
| done | done | ||
| Line 39: | Line 46: | ||
| # gives an error when it analyses the pdf file. | # gives an error when it analyses the pdf file. | ||
| gs -sDEVICE=pdfwrite -dNOPAUSE -dQUIET -dBATCH -sOutputFile=result.pdf $pdflst | gs -sDEVICE=pdfwrite -dNOPAUSE -dQUIET -dBATCH -sOutputFile=result.pdf $pdflst | ||
| - | rm out-*.pdf | + | rm $pdflst |
| exit 0 | exit 0 | ||
| Line 76: | Line 83: | ||
| ===== Scripting with unpaper ===== | ===== Scripting with unpaper ===== | ||
| + | ==== gray2black ==== | ||
| Automatic conversion from pdf files, containing gray images, to black and white images with unpaper and its noise reduction and cleaning up features can be done with the following script: | Automatic conversion from pdf files, containing gray images, to black and white images with unpaper and its noise reduction and cleaning up features can be done with the following script: | ||
| <code bash> | <code bash> | ||
| #!/bin/sh | #!/bin/sh | ||
| - | # filename: | + | # filename: |
| - | # This script | + | # This script |
| + | # unpaper, imagemagick and ghostscript to a black and white pdf file. | ||
| + | # | ||
| + | # Usage: gray2black input-file [option] output-file | ||
| # | # | ||
| - | # usage: findbt.sh < | + | # option: -b [value] specify black threshold value as being used in |
| + | # | ||
| + | # be used: -b 0.12 | ||
| + | # input-file: a pdf file containing sheetmusic with one or more gray | ||
| + | # | ||
| + | # output-file: | ||
| + | # settings will be created. | ||
| # | # | ||
| - | # < | + | # Example: |
| - | # < | + | # Process gray images in mozart.pdf: gray2black mozart.pdf -b 0.23 test.pdf |
| + | # | ||
| + | # This script uses imagemagick to convert and center an image on an | ||
| + | # a4 page. (With 2480x3508 pixels at a resolution of 300 dpi) and | ||
| + | # ghostscript to create a small pdf file. By using efficiently the | ||
| + | # piping method (|) in conjunction | ||
| + | # standard input (-) we get less tempfiles. | ||
| # | # | ||
| - | # Observe the outputfile.pdf and find the page which has the best value for black-threshold. (Text is embedded) | + | # NB. This script is time consuming. So have a lot of patience. |
| + | # Each page will take about 15 seconds to execute on a single core | ||
| + | # AMD64 3500+ cpu. | ||
| # | # | ||
| + | # User text | ||
| + | _fbhelp=" | ||
| + | _fbspdf=" | ||
| + | _fbcvrt=" | ||
| + | # Unpaper settings, start value for black-threshold is 0.12 | ||
| + | b_threshold=" | ||
| + | donot=" | ||
| + | filter=" | ||
| + | # Other settings | ||
| + | ptmpf=" | ||
| + | |||
| + | # Page dimensions | ||
| + | psize=" | ||
| + | horpix=2480 | ||
| + | verpix=3508 | ||
| + | pdim=" | ||
| + | pres=" | ||
| + | |||
| + | # Only resize if absolutely neccessary. Values for resizing are for left, right, top and bottom border. | ||
| + | hborder=33 | ||
| + | vborder=33 | ||
| + | lhorpix=$horpix | ||
| + | lverpix=$verpix | ||
| + | |||
| + | # Store input-file name temporarily, | ||
| + | arg1=$1 | ||
| + | |||
| + | # Find number of parameters passed | ||
| + | if [ " | ||
| + | echo " | ||
| + | exit 1 | ||
| + | fi | ||
| + | |||
| + | # Check OPTIONS | ||
| + | if [ " | ||
| + | case " | ||
| + | * ) | ||
| + | echo " | ||
| + | exit 1 | ||
| + | ;; | ||
| + | esac | ||
| + | elif [ " | ||
| + | case " | ||
| + | -b ) | ||
| + | shift | ||
| + | b_threshold=$2 | ||
| + | shift | ||
| + | ;; | ||
| + | * ) | ||
| + | echo " | ||
| + | exit 1 | ||
| + | ;; | ||
| + | esac | ||
| + | fi | ||
| + | |||
| + | # only process further if there are two arguments left. | ||
| + | if [ " | ||
| + | if [ ! " | ||
| + | echo " | ||
| + | exit 1 | ||
| + | else | ||
| + | # convert first page to a pgm file with filename $ptmpf-*.pgm | ||
| + | pdftoppm -gray -r 300 " | ||
| + | |||
| + | # Check for document resize neccessity. Iterating through all pages, makes sure, all pages get uniform resizing later. | ||
| + | |||
| + | resize=0 | ||
| + | # Iterate through files. We need to sort files in a numerical order because pdftoppm generates files without a leading zero. | ||
| + | # However before the numbers, a ' | ||
| + | for filenum in $(ls $ptmpf-*.pgm | cut -d' | ||
| + | dimension=" | ||
| + | # width: " | ||
| + | # height: " | ||
| + | if [ ${dimension%%x*} -gt $lhorpix ]; then | ||
| + | lhorpix=${dimension%%x*} | ||
| + | resize=1 | ||
| + | fi | ||
| + | if [ ${dimension## | ||
| + | lverpix=${dimension## | ||
| + | resize=1 | ||
| + | fi | ||
| + | done | ||
| + | # Resize document if neccessary | ||
| + | if [ $resize -eq 1 ]; then | ||
| + | # image fits within horizontal boundary, so adaptation must be vertical | ||
| + | if [ $horpix -eq $lhorpix ]; then | ||
| + | vscale=$(( ($verpix-30)*100/ | ||
| + | else | ||
| + | vscale=100 | ||
| + | fi | ||
| + | # image fits within vertical boundary, so adaptation must be horizontal | ||
| + | if [ $verpix -eq $lverpix ]; then | ||
| + | hscale=$(( ($horpix-30)*100/ | ||
| + | else | ||
| + | hscale=100 | ||
| + | fi | ||
| + | # scaling neccessary both horizontal and vertical | ||
| + | if [ $hscale -lt $vscale ]; then scale=$hscale; | ||
| + | |||
| + | echo Some images exceed the maximum size. Now scaling to " | ||
| + | # iterate through all files and resize all with the same percentage, use 8 bits per pixel | ||
| + | for filenum in $(ls $ptmpf-*.pgm | cut -d' | ||
| + | mogrify -depth 8 -resize " | ||
| + | echo resizing file: " | ||
| + | done | ||
| + | fi | ||
| + | # apply unpaper onto each page | ||
| + | for filenum in $(ls $ptmpf-*.pgm | cut -d' | ||
| + | # Parameter expansion: ${param%word} From the end remov? Mis je nog iets?es the smallest part of param that matches word and returns the rest. (p.72) | ||
| + | unpaper -b $b_threshold $filter $donot -t pbm " | ||
| + | rm " | ||
| + | done | ||
| + | |||
| + | # center pbm page on an a4 canvas, convert to pdf | ||
| + | pdflst="" | ||
| + | for filenum in $(ls $ptmpf-*.pbm | cut -d' | ||
| + | convert -size $pdim xc:white miff:- | composite -density $pres -units PixelsPerInch -compose atop -gravity Center " | ||
| + | rm " | ||
| + | pdflst=" | ||
| + | done | ||
| + | |||
| + | # merge all pages into on page | ||
| + | gs -sDEVICE=pdfwrite -dNOPAUSE -dQUIET -dBATCH " | ||
| + | rm $pdflst | ||
| + | fi | ||
| + | fi | ||
| + | exit 0 | ||
| + | </ | ||
| + | |||
| + | ==== findblack ==== | ||
| + | Find a value for black threshold with following script: | ||
| + | <code bash> | ||
| + | #!/bin/sh | ||
| + | # filename: findblack | ||
| + | # This script helps to find an unpaper' | ||
| + | # creating a series of pdf files each with a different black threshold setting. By observing | ||
| + | # the pages afterwards, selecting a proper black threshold value should be easier. Values are | ||
| + | # annotated on top of each page. | ||
| + | # | ||
| + | # usage: findblack input-file [option] output-file | ||
| + | # | ||
| + | # option: -p [number] which page should be used as source for unpaper. If not specified, first page will be taken. | ||
| + | # input-file: a pdf file containing one or more gray images. | ||
| + | # output-file: | ||
| + | # | ||
| + | # Example: | ||
| + | # Process page 4 from mozart.pdf: ./findblack mozart.pdf -p 4 test.pdf | ||
| # | # | ||
| - | # Use imagemagick to convert and center an image on an a4 page. (With 2480x3508 pixels at a resolution of 300 dpi). | + | # This script uses imagemagick to convert and center an image on an a4 page. (With 2480x3508 pixels at a resolution of 300 dpi) |
| - | # Use ghostscript to create a small pdf file. By using efficiently the piping method (|) in conjunction with | + | # and ghostscript to create a small pdf file. By using efficiently the piping method (|) in conjunction with |
| - | # reading and writing to the standard input (-) we get: | + | # reading and writing to the standard input (-) we get less tempfiles. |
| + | # | ||
| + | # NB. This script is time consuming. So have a lot of patience. | ||
| + | # Each page will take about 15 seconds to execute on a single core AMD64 3500+ cpu. | ||
| + | |||
| + | # User text | ||
| + | _fbhelp=" | ||
| + | _fbspdf=" | ||
| + | _fbcvrt=" | ||
| + | |||
| + | # Unpaper settings, start value for black-threshold is 0.05, end value for black-threshold is 0.40, step = 0.02 | ||
| + | a=5 | ||
| + | b=20 | ||
| + | s=5 | ||
| + | donot=" | ||
| + | |||
| + | # Other settings; $$ references the current PID | ||
| + | ptmpf=" | ||
| psize=" | psize=" | ||
| pdim=" | pdim=" | ||
| pres=" | pres=" | ||
| - | convert -size $pdim xc: | + | ptxt=" |
| - | # This will take about 5 seconds for a single page to execute on a single core AMD64 3500+ cpu. | + | pstr=" |
| + | # Default page number | ||
| + | defpage=1 | ||
| + | |||
| + | # Find number of parameters passed | ||
| + | if [ " | ||
| + | echo " | ||
| + | exit 1 | ||
| + | fi | ||
| + | |||
| + | # Store input-file name temporarily, | ||
| + | arg1=$1 | ||
| + | |||
| + | # Check OPTIONS | ||
| + | if [ " | ||
| + | case " | ||
| + | * ) | ||
| + | echo " | ||
| + | exit 1 | ||
| + | ;; | ||
| + | esac | ||
| + | elif [ " | ||
| + | case " | ||
| + | -p ) | ||
| + | shift | ||
| + | defpage=$2 | ||
| + | shift | ||
| + | ;; | ||
| + | * ) | ||
| + | echo " | ||
| + | exit 1 | ||
| + | ;; | ||
| + | esac | ||
| + | fi | ||
| + | |||
| + | # only process further if there are two arguments left. | ||
| + | if [ " | ||
| + | if [ ! " | ||
| + | echo " | ||
| + | exit 1 | ||
| + | else | ||
| + | # find last page, start counting from 0, necessary for proper deleting pdftoppm result file which may have extra 0 | ||
| + | trail="" | ||
| + | lastPage=" | ||
| + | if [ " | ||
| + | trail=" | ||
| + | elif [ " | ||
| + | trail=" | ||
| + | elif [ " | ||
| + | trail=" | ||
| + | elif [ " | ||
| + | trail=" | ||
| + | fi | ||
| + | # convert first page to a pgm file with filename tmpfile_PID-1.pgm | ||
| + | pdftoppm -gray -r 300 -f $defpage -l $defpage " | ||
| + | # check for succesful conversion to .pgm | ||
| + | if [ ! -f " | ||
| + | echo " | ||
| + | rm " | ||
| + | exit 1 | ||
| + | else | ||
| + | pdflst="" | ||
| + | |||
| + | nmb=$a | ||
| + | while [ " | ||
| + | do | ||
| + | # Cope with digits | ||
| + | if [ " | ||
| + | str1=" | ||
| + | else | ||
| + | str1=" | ||
| + | fi | ||
| + | |||
| + | # process unpaper | ||
| + | unpaper -b $str1 $donot -t pbm $ptmpf-$trail$defpage.pgm $ptmpf.pbm | ||
| + | |||
| + | # Create white A4 size canvas -> center pbm file into canvas -> convert pbm to ps -> convert ps to pdf | ||
| + | # New to bash? | ||
| + | # Send to and retreive from ' | ||
| + | # Sending result to next command with pipe: | | ||
| + | # Using this ' | ||
| + | convert -size $pdim xc: | ||
| + | |||
| + | pdflst=" | ||
| + | nmb=$(($nmb+$s)) | ||
| + | rm $ptmpf.pbm | ||
| + | done | ||
| + | rm $ptmpf-$trail$defpage.pgm | ||
| + | # Merge all pdf files in $pdflst into one single file with filename $2 | ||
| + | gs -sDEVICE=pdfwrite -dNOPAUSE -dQUIET -dBATCH -sOutputFile=$2 $pdflst | ||
| + | rm $pdflst | ||
| + | fi | ||
| + | fi | ||
| + | fi | ||
| + | exit 0 | ||
| </ | </ | ||
| + | |||
| + | ===== Bugs ===== | ||
| + | [[software: | ||
software/unpaper_test.1236727283.txt.gz · Last modified: 2009/03/11 00:21 by admin
Page Tools
Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Noncommercial-Share Alike 4.0 International
