Slides used by Dave Sanderson (Auckland War Memorial Museum) and Mark Beatty (National Library of New Zealand) for Digitisation 101 workshop at NDF (National Digital Forum) 2016.
2. Dave Sanderson – Project Leader Collection Imaging
@thebiglankyape
dsanderson@aucklandmuseum.com
Mark Beatty – Imaging Technician & Photographer
@MarkBTea
mark.beatty@dia.govt.nz
3. Programme for the morning session:
9 – 10:30 Introductions, expectations for the day and
session no. 1
Project planning and setup considerations
10:30 – 11 Morning Tea
11 – 12:00 Session no. 2
Imaging standards, formats and equipment
selection (assessing needs for collection type)
12:00 – 1:00 Lunch
4. Programme for the afternoon session:
1:00 – 2:30 Practical Session
2:30 – 3 Afternoon tea
3 – 3:30 Practical session Q&A
3:30 – 4 Exporting and publishing of your data
4 onwards ??? – earlier finish
5. Project planning lessons from Archives NZ and AWMM
You’ve got to ask the obvious question –
Do you need to digitise it at all? If so,
why?
6. Project planning lessons from Archives NZ and AWMM
Then you have to get into that boring
stuff…
Scope – know before you start how big your project is and
define your boundaries
Funding – far too easy to digitise ‘things’ and not factor all of
the costs in
7. Funding considerations
Equipment (and its replacement)
Staffing
Software (largely subscription based)
On-going costs to maintain the files you’re creating ‘forever’
If it’s not forever, the cost of decomissioning
OPEX vs CAPEX – is it an asset and collection in its own right?
8. Project planning lessons from Archives NZ and AWMM
You’re a much easier sales pitch if you
align with your organisational strategy…
9. Project planning lessons from Archives NZ and AWMM
Test (thoroughly)
Good sample selections – this is
particularly relevant for lighting on 3D
collections
Be accurate, not over or under
10. Project planning lessons from Archives NZ and AWMM
Set targets
Make them realistic
Balance between numbers
watching/doing ‘just enough’ and
leading digitising staff – motivation &
leadership are key
11. Project planning lessons from Archives NZ and AWMM
At AWMM you’d struggle to find
someone who doesn’t know what
Future Museum is and how their work
supports it
29. Standards, formats and equipment - AWMM
Simple – all digitisation is photography,
so go big!
Canon EOS 5Ds / 5DsR – both 50MP and
just under 9000px
30. Standards, formats and equipment - AWMM
Now implementing DAMS (Piction) which is forcing
the hand in coming up with a museum-wide
standard
Consolidated approach for all photography by all
staff in all situations. Has significant impact on
external partners
Includes video and audio files as well as specialist
files
31. Standards, formats and equipment - AWMM
Ask yourselves – are you going to keep your ‘stuff’
(going back to the why are you doing this question)
Are you going to start to do digital preservation?
Major driver for digitsation standards!
32. Standards, formats and equipment - AWMM
We save 4x file formats for each image:
RAW (Canon CR2)
DNG (Adobe proprietary RAW)
Vanilla 8-bit Uncompressed TIFF
JPEG (and derivatives)
Metadata – get your metadata sorted!
NO Photoshop & destructive editing
33. Standards, formats and equipment - AWMM
Look for JHOVE and DROID if you want
to get geeky and validate your files
Look into PREMIS metadata schema
34. Standards, formats and equipment - AWMM
Look for JHOVE and DROID if you want
to get geeky and validate your files
Look into PREMIS metadata schema
35. Standards, formats and equipment - AWMM
We’re doing some cool stuff with 35mm
slides soon!
36. Imaging Services – Standards / Formats
• Early years
• Guided by international standards FADGI/Metamorfoze
• Dependent on size of item
• Preservation Master / Modified Master
• Access derivatives– jpeg, jpeg 2000
• Filenaming
• Colour management
37. Printed Text Pictorial Materials Oversized Materials Manuscripts
Examples of
formats
Periodicals,
newspapers,
serials, books and
journals
Photographic prints
and negatives,
paintings, drawings
and prints, ephemera
and cartoons.
Maps, posters, and
manuscripts (E.g.
Whakapapa charts).
Letters, diaries,
journals, notebooks,
and scrapbooks.
Minimum requirements
File format TIFF 6.0 ITU G4
(compressed)
TIFF 6.0
(uncompressed)
TIFF 6.0
(uncompressed)
TIFF 6.0
(uncompressed)
Capture
Resolution
400ppi 3,000 pixels on the
longest dimension
300pp4i 300ppi
Bit depth 1-bit
black & white
8-bit greyscale
(photographs)
8-bits per colour
(24-bit RGB)
8-bits per colour
(24-bit RGB)
8-bit greyscale
(Digitised from
microfilm)
Colour space
ICC profile
N/A Gray Gamma 2.2
(Greyscale)
Adobe 1998 (Colour)
Gray Gamma 2.2
(Greyscale)
Adobe 1998 (Colour)
Gray Gamma 2.2
(Greyscale)
Adobe 1998 (Colour)
Recommended requirements
File format TIFF 6.0
(uncompressed)
TIFF 6.0
(uncompressed)
TIFF 6.0
(uncompressed)
TIFF 6.0
(uncompressed)
Capture
Resolution
400ppi Minimum of 5,000
pixels on the longest
dimension
300ppi 300ppi
Bit depth 8-bit greyscale
or
16-bits per colour
(48-bit RGB)
16-bits per colour
(48-bit RGB)
16-bits per colour
(48-bit RGB)
16-bits per colour
(48-bit RGB)
Colour space
ICC profile
Gray Gamma 2.2
(greyscale)
Adobe 1998
(colour)
Adobe 1998 (Colour) Adobe 1998 (Colour) Adobe 1998 (Colour)
38.
39.
40. Questions from you relating to standards, formats and
equipment?
Open discussion on how we’d all approach the solutions
41. Resources
• https://pinboard.in/u:imagingservices
• Digital NZ- http://www.digitalnz.org/make-it-digital
• http://www.digitizationguidelines.gov/guidelines/digitize-technical.html
• https://www.metamorfoze.nl/english/digitization
• http://www.digitizationguidelines.gov/guidelines/digitize-planning.html
• http://en.nationaalarchief.nl/sites/default/files/docs/guidelines_digitisation_phot
ographic_materials_0.pdf
• https://luminous-landscape.com/scannerless-digital-capture-and-processing-of-
negative-film-photographs/
• https://www.youtube.com/watch?v=miI4kcH_S1g&feature=youtu.be
• The AIC Guide to Digital Photography and Conservation Documentation
• http://imagemuse.org/
• NZ Photographers of Cultural Collections
46. Exporting and publishing of data – AWMM
Capture: DSLR Remote Pro
Import, Process and Export: Adobe Lightroom
No editing done beyond final review of white balance and embedding of
a series of tags:
1) Auckland War Memorial Museum
2) Collection Department (eg Natural Sciences)
3) Collection Area (eg Marine)
4) Photographer name (eg Jennifer Carol)
47. Exporting and publishing of data – AWMM
Remember no Photoshop! You’ve got to
edit your files very carefully if you want
them to survive long term.
48. Exporting and publishing of data – AWMM
File formats for the web:
Almost always JPEG – JPEG2000 if you want to do something clever.
Set your resolution accordingly (72ppi / 96ppi)
Think about how ‘big’ your image wants to be? 1000px, 2000px, full res?
Please do not watermark your images – very out of date
Looking into IIIF for universal sharing of images (http://iiif.io/ )
How do you want to share and license? Digital New Zealand and Creative
Commons
Usage and Image rights go hand-in-hand with digitisation!
49. Exporting and publishing of data – AWMM
Ask yourself how proprietary your
platform is?
Can you move on?
Does it share via APIs? (Archives NZ (no) vs
AWMM (yes))
50. Exporting and publishing of data – AWMM
If you’re doing crowd sourcing or user
contributed content – know before you
start if you plan to keep the info, and
how you plan on treating it
51. Imaging Services – Export and Publishing
• Part of processing workflow – export via Capture one, Photoshop or
Lightroom
• Colour management – BasicColor – IQ Analyzer
• Data managed by National Digital Heritage Archive
https://digitalpreservation.natlib.govt.nz/
• Collections deposited using Indigo
• Metadata
• Descriptive records –Tiaki/ Emu
• Web team manage online access
• Digital NZ
• Public orders
May be a bit of cross over between my comments and Davids slides, but I am giving NLNZ perspective.
My experience – 10 years – copying as a service, now increasing demand for collection items to be digitised – NLNZ strategy document mentions this frequently.
We are boutique and not mass, yet demand is moving in this direction. Programme work, projects that arise, specific collections being identified, opportuntites related
to historical celebrations….
Key thing is to have a plan, and follow it as best you can.
Think careful around reasons for doing this work, easy to be blinded by high expectations – what is achievable with resources available
How do you locate funding? What are risks, challenges – in-house v external provider- More control in house, compromises on both sides.
Consider collections, have one already to go, with good records, conservation completed, less resources needed.
If collections are restricted to on-site viewing why digitise ? All persons at library can view. They can request original anyway!
Commitment of staff time needs to be identified across teams. Existing staff what do they need to do, Conservation, A&D, IT.
Define timeframe, budget restricts. Over runs, extension without resources will result in BAU resolving, can be a headache.
“Capture once, use many times” Standards can be copied, or you define your own by adjusting to your needs – eg – PM / MM 6 bit v 8 bit – Tiff v jpeg 2000 – colour managing -
Equipment options a plenty – just scope what your budget allows. Lots of advice you can get.
IT, server needs. Get in early with requirements to allow for having in place.
Good idea to test workflows, have a flexible approach in first stage/ quarter. Correct and improve if possible.
Quality assurance, Control essential , will save in long term.
Key to have trainer who can easily share workflows, and engage with project worker so they become champions of collections
Existing H-series war negatives collection completed a number of years ago – Now free to download
WWI Music scores, existing Programme work and public order demands
Manuscripts curator – David Colquhoun / Sean McMahon– digitise to make accessible off site.
Timeline –
David Adams, Imaging Services Co-ordinator
Recruitment – Ish Dooney – Graduate of Massey Photography degree program
Limited amount of conservation work – Most diaries already boxed thanks to previous work.
Legacy equipment form pictures online project used. Acquired new lights.
Claire involved in training Ish over number of weeks. Use of presets, actions to speed things up.
Imaging Services will likely lead such projects in future.
Again good recruitment, meant Ish was very invested in work she was doing. Wrote some articles
And a blog.
28,000 pages from soldiers’ diaries and private letters, digtised – 70 items –
At end of project managed to extend Ish contractto complete some Donor copying
Use of dual screens, essential supports used
Assortment of items digitised as part of WWI Diaries project
Ish Doney – at setup used- Fingers instead of glass
Image showing collection of front covers, letters digitised as part of War diaries project. This image was used in
WWI exhibtion, book cover, as part of WW100 remembrance at memorial park
Pre-existing donor agreements required this work to be completed.
Work had started in 2014 on Stout papers – but not completed.
Not BAU – support from Crown ….. asset
Project space in place – copy stand+ equipment available.
Recruitment – good to have casuals – if possible – time limits trianing, so need self starter who picks things up quickly.
Identify what is desirable in candidate – Photography background, attention to detail, reptitive nature of work etc
New workflow- agreed standard with Curator –
Used Lightroom, direct import with Dslr remote pro in live view to ease alignment and enable instant QA as work done.
Presets to process – Good working excel sheet for Project Photographer to tick off steps as completed.
By second week Camus working independently – close contact with trainer + MS curator + curator
Nature of folders, meant underestimation of numbers blow out from 4600 to over 6000
Collection of photographs which needed processing under our standards with PM and MM –
Project extended by 5 weeks – we still ran out of time – so tidy up to be completed by Imaging as part of BAU.
Outcomes – papers and photographs entirely copied – PDFs supplied to donors –
Camus was so invested in project that he completed blog in relation to WW100 and Rudolph Gray.
We adjusted some of our standards to fit project needs and speed. Fixed height of camera. Use of Canon 5DS for
Photographs.
Camus Wyatt – Project Photographer – Donor copying project
Background as documentary photographer, some experience in dealing with archives
Portrait of Anna Paterson Logan (later Stout), 9 July 1874 Ref: PAColl-10309-02-01
MS-Papers-11518-103_002 - Letter from Sir Robert Stout to his wife
MSX-9368 – Letter Book 2 Randolph Norman Gray - Letter book by Gray describing the time spent stationed at the military camp at Heliopolis near the pyramids in Egypt, including marches, visits to local markets, guided camel tours and other excursions during periods of leave.
MS-Papers-11756-5-086-Correspondence, chiefly in German, relating to efforts made by Helmut to emigrate from Germany to a country that was willing to accept him. During this time Helmut made efforts to migrate to Sweden, United States, United Kingdom, Australia and New Zealand.
Ref: PAColl-10493-3-16
Carte de visite portrait of Thomas Duncan Macgregor Stout as a child, between 1889 and 1890. Photograph taken by David Alexander De Maus (Port Chalmers) during New Zealand and South Seas Exhibition at Dunedin.
Ref: PAColl-10309-12-17
Studio portrait of Thomas Duncan Macgregor Stout in military uniform during World War Two, circa 1940-1945. Photographer unidentified.
Established in 1990s – not much to compare with – standards defined primarily by equipment abilities.
Legacy files vary greatly – Scanning from analog to digitise – so inconsistent colours due to film & nature of how
they were digitised at time. Ok to update change.
“Capture once, use many times” - were mistakes made in this period – were collections digitised with short use in mind – web only
Establishment of standards by larger institutions, Library of Congress,NARA British Library, Getty etc.
Equipment changes, Betterlight scanning back, High quality monitors- Gretagmacbeth, Xrite monitor profiling -
2005 NLNZ Imaging Standards – followed existing guidelines –
2013 – review of output standards only – some changes – programme digitisation – eg Manuscripts – MM only kept.
We don’t outline based on Dpi/ PPI as confusion caused as this will vary depending on format of item and scale.
Link to Imaging Standards –
What fits your resources – what is maximum possible –
Steer clear of propriety file formats – even though we shoot in raw format – use tiff as our digital preservation master format
Derivatives created by Rosetta, automated task, system creates. This can be actioned as part of your processing of files – Pictures online
Used this approach.
Standards should also address how files should be edited, processed. Use of targets, how objects are photographed, named,
sharpened.
Slide showing our standards document – 2006 document – which has since been updated in 2013 to account for
Larger formats and Manuscript digitisation – principles the same – some changes with regard to manuscripts and larger format items.
FADGI – Eg Reproduction of Artwork – Star from 1 to 4 – Metamorfoze takes similar approach
Equipment
Top Left: Canon DSLR 5D Mk III cameras, small studio capture and corporate/event photography (approx. 22 megapixel camera)
Top Right: Zeutschel overhead scanner – used for our research copying by our Library Assistants – able to capture 2 pages at once,
Works well for bound items
Btm Left:
Nikon Supercoolscan 5000 ed (35mm negative scanner) – move towards lightbox setup for larger items
Btm Centre: Phase One - IQ180 (back - sensor) with 645 DF+ plus (camera body). It has a sensor which captures 80 megapixel files.
Btm Right: Epson v700 flatbed scanner. We can capture 16 bit RGB colour files, file size up to about 700 mgb (variable ppi depending on size of original).
For a 12 MP camera, you can have 9 x 14” (23 x 35 cm) @ 300ppi (bit larger than A4) , or 14 x 21” 35 x 53 cm (almost A2 size) @200ppi print.
Software –
Pinboard: I update this regular and you can search
Digital NZ – undergoing review
Archives NZ are also reviewing there standards which are aimed at Government
FADGI – Federal agencies digitization guidelines initiative- Project planning from 2009 but still very relevant
Metamorfoze – National Library of the Netherlands
FADGI – Project planning quite comprehensive in approach
Usefulll guide to negative materials from National Archives of Netherlands
One form Luminous landscape – downloadable PDF which I found useful
Of course Youtube is great especially for seeing conference talks
American Institute of Conservation produced this very useful book
Forums – as questions find out stuff
Export and Publishing – Mark
Workflow leads to PM/MM export – then deposit using Indigo –
Colour management – new approach to be implemented shortly – Will allow profiling of all our setups
and regular checks to be done on results achieved. More numbers based.
Prior to establishment of NDHA and attendant support – We had an object management system –
Uncontrolled access – no control over changing of data present.
Digital Preservation of paramount importance, not just for in house digitisation but for published, unpublished digitally born
What options for smaller institutions – Vernon
NDHA – Rosetta – Ex-Libris – Long term care of digitised collections.
Metadata relates to camera, format, reason for digtisation. Limited use presently. All exif data maintained in files.
A&D – records need to be matched through indigo .
Digtial NZ Api –
Public order – easy availability of collections digitised – Desire to provide collections for free where no know copyright –
Otherwise order directly – system generated jpegs supplied, released by Librarians once no copyright issues.
Managed by copying coordinator & Library Assistant
Items requiring digitisation handled within 10 -14 working days. -