Target and Process Policies Processing Tab

Previous Next  Print this Topic

Use the Processing tab to define settings for how you want to process files or video. The contents on the Processing tab are the exact same as displayed in User Preferences. The preferences entered in at the Processing tab override any defaults that may be set in the User Preferences window, but only for the current job or policy.

Target and Process Policy - Processing tabFile Processing Options

Use to set processing options for permissions, file processing, and metadata processing.

Elements

Description

Override original file permissions to allow any user to retrieve

Phoenix EXP adheres to the level of rights and access control that is set for the operating system where files are  ingested from. Typically, only the owner, author of the files or a system administrator with administrative or root level access privileges is able to view, open, and retrieve files in the Information Repository.

Select to allow access on stored files so any user can retrieve files.

This feature negates all protection associated with your files; anyone on your network using Phoenix EXP can access your stored files.

File Grouping Processing

Active only when Phoenix EXP plug-ins are implemented, this option determines how ingested files are grouped and recognized by search clients. Use this feature to group non-video files with video files so search results include all associated files.

Select from options:

None - No file grouping.

Files in each folder together - Groups files by folder so  all of the files in any given ingested folder are in a single group.

All files together - Groups all of the files in a job into one group, regardless of what folder they are in.

For example, if ingesting a video file, a metadata file for the video, and an .stx file for the video from the same folder, and you want them all to be returned when searching for just the video file, you would select Files in each folder together; if they are in disparate folders, you would select All files together.

File grouping does not affect the folder structure seen in Search and Retrieve.

Extended Metadata Processing

Add custom metadata to files that are being ingested. In contrast to the Metadata tab, which allows you to manually add metadata during ingest processing, the Extended Metadata Processing allows you to programmatically add metadata, on a per directory or per file basis, during ingest processing.

Ingest.ExtMetadata, <filename>.md, and script files have diverse purposes, select from options:

None - No extended metadata processing.

Use Ingest.ExtMetadata - Affects files in its own folder and child folders. If a child folder has its own Ingest.ExtMetadata, files in that folder and its child folders use it instead of a Ingest.ExtMetadata in a parent folder.

Use '.md' file - Adds metadata only to files that have an identical filename and reside in the same folder. For example, if a .md file is created and named films.md and it is placed in a folder in primary storage, metadata would be added, at ingest, only to the file named films.doc in the same folder.

Use script - Adds metadata based on a user-provided script file located in the Config folder in the Phoenix installation directory. The file must be named UserExtMdProc.bat (for Windows) or UserExtMdProc.sh (for Linux/Mac). This script file will create an XML file for the metadata values that will be added at ingest with the files being ingested.

 

Setup for Extended Metadata Processing

Using Ingest.ExtMetadata and <filename>.md

There are three different delimiters available for use when creating a Ingest.ExtMetadata or a <filname>.md file. You can use a:

Equal sign: <token>="<value>"
Tab: <token>        "<value>"
Comma: <token>,"<value>"

Delimeters must be consistent within the file and cannot be mixed. The first delimiter found (equal sign, tab, or comma) will be used as the delimiter when parsing the rest of the file. The following example uses an equal sign (=) as the chosen delimeter.

DVD_Title="The Passion of the Christ"

Actors="James Caviezel Monica Bellucci Maia Morgenstern Christo Jivkov Francesco De Vito"

Directors="Mel Gibson"

Writers="Mel Gibson Benedict Fitzgerald"

Producers="Mel Gibson Bruce Davey Enzo Sisti Stephen McEveety"

Format="AC-3 Closed-captioned Color Dolby DTS Surround Sound DVD Full Screen Subtitled NTSC"

Language="Hebrew"

Subtitles="English Spanish"

Region="1"

Aspect_Ratio="1.33:1"

Number_discs="1"

DVD_Release_Date="2004/08/31"

Run_Time="127 minutes"

Tokens may not contain spaces, and values must be enclosed in quotation marks.

When using an equal sign as a delimiter, there can be no white space before or after the equal sign.

If you use multiple values for a single token and want to find a single value using a Phoenix EXP search application, search using "contains". If you searched using "equals", only results that match the full string of multiple values would be returned.

Follow these steps to implement an extended metadata Ingest.ExtMetadata or <filename>.md file:

1.Create the Ingest.ExtMetadata or <filename>.md file using any text editor.
2.Name it either Ingest.ExtMetadata or <filename>.md depending on the functionality you want.
3.Copy Ingest.ExtMetadata to a folder where all files have associated metadata. Copy <filename>.md to folders where a specific file, named identically to <filename>, is to have associated metadata.
4.Select an extended metadata option from the Extended metadata processing drop-down list on the Processing tab.

Using Script

Selecting the option Use script for extended metadata processing will add metadata based on a user-provided script file. There are two phases to processing extended metadata: Process and Finalize. In each phase, certain parameters are passed to the user-provided script file in a file in the following format:        

<EmdScriptParameters>

   <Phase>Process</Phase>

   <TargetFile>/file/being/stored</TargetFile>

   <IsContainer>No</IsContainer>

   <OutMdFile>/location/of/output.xml</OutMdFile>

</EmdScriptParameters>

Phase = Process or Finalize

TargetFile = The file being stored in the Information Repository

IsContainer = Yes or No and indicates whether the target file is a container (directory) or a file

OutMdFile = The location of the XML output file that the script will generate

The above example is in the first phase, Process. When Use script is chosen on the Processing Tab of Target and Process Policies, Target and Process Policies calls the user-provided script and passes it a file with the arguments listed in the example above. The script needs to create an XML file for the metadata in key/value pairs. These key/value pairs will be contained within ExtendedMetadata tags as shown below.

<ExtendedMetadata>

   <Key>Value</Key>

   <Key>Value</Key>

   <Key>Value</Key>

</ExtendedMetadata>

The XML output file needs to be created in the location specified by the OutMdFile parameter that was passed to the script by Target and Process Policies.

After the XML file has been created by the user-provided script and the Process phase is complete, Target and Process Policies will call the script again with a new file. Now it is in the Finalize phase and the preliminary XML metadata values will be included in the EmdScriptParameters block that is passed to the script.

<EmdScriptParameters>

   <Phase>Process</Phase>

   <TargetFile>/file/being/stored</TargetFile>

   <IsContainer>No</IsContainer>

   <OutMdFile>/location/of/output.xml</OutMdFile>

 

   <ExtendedMetadata>

       <Key>Value</Key>

       <Key>Value</Key>

       <Key>Value</Key>

   </ExtendedMetadata>

 

</EmdScriptParameters>

At this point during the Finalize phase, the user-provided script will have a chance to normalize the preliminary metadata values that were included within the EmdScriptParameters block and make any corrections or changes if needed. The script will create a new XML file with the final metadata values at the location specified by the OutMdFile parameter that was passed to the script by Target and Process Policies during the Finalize phase. The final XML file at the location specified by the OutMdFile parameter will be used by Target and Process Policies to create metadata that will be processed and included with each file or folder that is being ingested.

To see an example script file, go to http://www.SoleraTec.com and visit the Phoenix EXP Resources page.

Follow these steps to implement an extended metadata Script file:

1.Create the batch/script file that will produce metadata in an XML format (as shown above).
2.Name the existing file UserExtMdProc.bat (Windows) or UserExtMdProc.sh (Linux or Mac).
3.Place the file in the Config folder in the Phoenix installation directory.
4.Select the Use script option from the Extended metadata processing drop-down list on the Processing tab.