Datastage Desiner的快捷键
1 关闭某个job <Ctrl+F4>
2 <Ctrl+F6>同时打开了两个以上job，在不同的job之间切换。
Datastage Director的快捷键

1 First you should already have Tortoise installed.
2 Download the plugin here:http://www.tabaquismo.freehosting.net/ignacio/eclipse/tortoise-svn/subversion.html
3 Unzip the plugin to the plugins directory in your Eclipse as u usually do.
4 Now open the Preference page of your Eclipse and you can see Tortoise if everything goes well.
5 click Tortoise and you have three fields to configure,the following is what I configured :
Path:C:\Program Files\TortoiseSVN\bin\TortoiseProc.exe
SVN Server:128.64.181.66
Shared Root Folder:/CARTCCP/CARTCCP/proj_/,your project's parent dir.

Now you can File--Import--Existing Eclipse projects into Workspace,choose your project and import.

Till now you can right-click on any files in your project and you can see shortcut menu Tortoise and here you can do update、commit and so on.

#!/bin/ksh
#######
################################################################################
#######
####### FILE: MasterControl.ksh
#######
####### DESCRIPTION: Starts a DataStage MasterControl type job passing
#######              all runtime parameter values
#######
#######
####### Date       Version   Developer      Description
####### ---------- --------- -------------- ------------------------------------
####### 2002-05-15 1.0       Ken Bland      Initial release
####### 2002-06-27 2.2       Ken Bland      FileSetDate/ProcessDate modifications
####### 2002-07-10 2.2       Steve Boyce    Added ProcessDate as 14th parameter
####### 2002-08-16 2.3       Steve Boyce    Now calls stored procedure
#######                                     GET_NEW_BATCH_NBR in datastage schema
#######                                     instead of deriving it and inserting
#######                                     here.
#######                                     Uses comSQLPlus.ksh and comPLSQL.ksh
#######                                     instead of SQLPlusStub.ksh.ksh
#######

PROG=`basename ${0}`
EXIT_STATUS=0

NOW=`date`
echo "${NOW} ${PROG} Initialization..."
echo

#######
####### CONFIGURATION ##########################################################
#######

if [ ${#} -ne 14 ]; then
   echo "${NOW} ${PROG} : Invalid parameter list."
   echo "${NOW} ${PROG} : The script needs 14 parameters:"
   echo "${NOW} ${PROG} :    JobName"
   echo "${NOW} ${PROG} :    ParameterFile"
   echo "${NOW} ${PROG} :    FileSetDate (YYYY-MM-DD)"
   echo "${NOW} ${PROG} :    BatchNumber"
   echo "${NOW} ${PROG} :    JobHierarchyFile"
   echo "${NOW} ${PROG} :    SourceSystemList"
   echo "${NOW} ${PROG} :    SubjectAreaList"
   echo "${NOW} ${PROG} :    ClearWorkArea"
   echo "${NOW} ${PROG} :    StartingMilestone"
   echo "${NOW} ${PROG} :    EndingMilestone"
   echo "${NOW} ${PROG} :    DebugMode"
   echo "${NOW} ${PROG} :    JobLinkStatisticChecksFile"
   echo "${NOW} ${PROG} :    ResurrectLogFile"
   echo "${NOW} ${PROG} :    ProcessDate (NULL|YYYY-MM-DD H24:MI:SS)"
   exit 99
fi

JobName="${1}"
ParameterFile="${2}"
FileSetDate="${3}"
BatchNumber="${4}"
JobHierarchyFile="${5}"
SourceSystemList="${6}"
SubjectAreaList="${7}"
ClearWorkArea="${8}"
StartingMilestone="${9}"
EndingMilestone="${10}"
DebugMode="${11}"
JobLinkStatisticChecksFile="${12}"
ResurrectLogFile="${13}"
ProcessDate="${14}"

echo "${NOW} ${PROG} JobName ${JobName}"
echo "${NOW} ${PROG} ParameterFile ${ParameterFile}"
echo "${NOW} ${PROG} FileSetDate ${FileSetDate}"
echo "${NOW} ${PROG} BatchNumber ${BatchNumber}"
echo "${NOW} ${PROG} JobHierarchyFile ${JobHierarchyFile}"
echo "${NOW} ${PROG} SourceSystemList ${SourceSystemList}"
echo "${NOW} ${PROG} SubjectAreaList ${SubjectAreaList}"
echo "${NOW} ${PROG} ClearWorkArea ${ClearWorkArea}"
echo "${NOW} ${PROG} StartingMilestone ${StartingMilestone}"
echo "${NOW} ${PROG} EndingMilestone ${EndingMilestone}"
echo "${NOW} ${PROG} DebugMode ${DebugMode}"
echo "${NOW} ${PROG} JobLinkStatisticChecksFile ${JobLinkStatisticChecksFile}"
echo "${NOW} ${PROG} ResurrectLogFile ${ResurrectLogFile}"
echo "${NOW} ${PROG} ProcessDate ${ProcessDate}"
echo

# Below will look in the parameters.ini file to determine the directory path each.
UserID=`whoami`
BinFileDirectory=`cat /.dshome`/bin
LogFileDirectory=`grep -w LogFileDirectory ${ParameterFile}|cut -d "=" -f2`
TempFileDirectory=`grep -w TempFileDirectory ${ParameterFile}|cut -d "=" -f2`
CommonScriptFileDirectory=`grep -w CommonScriptFileDirectory ${ParameterFile}|cut -d "=" -f2`
CommonLogFileDirectory=`grep -w CommonLogFileDirectory ${ParameterFile}|cut -d "=" -f2`
LogFileName=${CommonLogFileDirectory}/${PROG}_${JobName}.log
TEMPBATCHNBRLOG=${TempFileDirectory}/${PROG}_${JobName}_start.log
DATASTAGEPROJECT=`grep -w DATASTAGEPROJECT ${ParameterFile}|cut -d "=" -f2`
DSSERVER=`grep -w DSSERVER ${ParameterFile}|cut -d "=" -f2`
DSUSERID=`grep -w DSUSERID ${ParameterFile}|cut -d "=" -f2`
DSPASSWORD=`grep -w DSPASSWORD ${ParameterFile}|cut -d "=" -f2`

NOW=`date`
echo "${NOW} ${PROG} UserID ${UserID}"
echo "${NOW} ${PROG} BinFileDirectory ${BinFileDirectory}"
echo "${NOW} ${PROG} LogFileDirectory ${LogFileDirectory}"
echo "${NOW} ${PROG} TempFileDirectory ${TempFileDirectory}"
echo "${NOW} ${PROG} CommonScriptFileDirectory ${CommonScriptFileDirectory}"
echo "${NOW} ${PROG} CommonLogFileDirectory ${CommonLogFileDirectory}"
echo "${NOW} ${PROG} LogFileName ${LogFileName}"
echo "${NOW} ${PROG} TEMPBATCHNBRLOG ${TEMPBATCHNBRLOG}"
echo "${NOW} ${PROG} DATASTAGEPROJECT ${DATASTAGEPROJECT}"
echo "${NOW} ${PROG} DSSERVER ${DSSERVER}"
echo "${NOW} ${PROG} DSUSERID ${DSUSERID}"
echo "${NOW} ${PROG} DSPASSWORD *Protected*"
echo

#######
####### PARAMETER BUILD Without batch number ##################################
#######

if [ "${ProcessDate}" = "NULL" ]; then
   StartTimestamp=`date '+%Y-%m-%d %H:%M:%S'`
else
   StartTimestamp="${ProcessDate}"
fi
ParamList=" -param ParameterFile=${ParameterFile}"
ParamList="${ParamList} -param ProcessDate=\"${StartTimestamp}\""
ParamList="${ParamList} -param FileSetDate=${FileSetDate}"
ParamList="${ParamList} -param JobHierarchyFile=${JobHierarchyFile}"
ParamList="${ParamList} -param SourceSystemList=${SourceSystemList}"
ParamList="${ParamList} -param SubjectAreaList=${SubjectAreaList}"
ParamList="${ParamList} -param ClearWorkArea=${ClearWorkArea}"
ParamList="${ParamList} -param StartingMilestone=${StartingMilestone}"
ParamList="${ParamList} -param EndingMilestone=${EndingMilestone}"
ParamList="${ParamList} -param DebugMode=${DebugMode}"
ParamList="${ParamList} -param JobLinkStatisticChecksFile=${JobLinkStatisticChecksFile}"
ParamList="${ParamList} -param ResurrectLogFile=${ResurrectLogFile}"

#######
####### Get Batch Number and create ETL_BATCH_AUDIT record #####################
#######

echo "${NOW} ${PROG} About to get new BATCH_NBR and insert it into ETL_BATCH_AUDIT..."
${CommonScriptFileDirectory}/comPLSQL.ksh ${ParameterFile} "IRDSN" "IRUserID" \
                                           GET_NEW_BATCH_NBR \
                                              "${JobName}" \
                                              "${StartTimestamp}" \
                                              "${UserID}" \
                                              "${SourceSystemList}" \
                                              "${SubjectAreaList}" \
                                              "${ParamList}" \
                                              "${FileSetDate}" > ${TEMPBATCHNBRLOG}
SQL_EXIT_STATUS=$?
cat ${TEMPBATCHNBRLOG}
if [ "${SQL_EXIT_STATUS}" != 0 ]; then
   NOW=`date`
   echo "${NOW} ${PROG} Failure to connect/insert into ETL_Batch_Audit table!"
   exit ${SQL_EXIT_STATUS}
fi

#######
####### Get BATCH_NBR from batch number log file ##############################
#######
BatchNumber=`grep -w BATCH_NBR ${TEMPBATCHNBRLOG}|cut -d "=" -f2`
if [ -z "${BatchNumber}" ]; then
   NOW=`date`
   echo "${NOW} ${PROG} Failure to retrieve BATCH_NBR from ${TEMPBATCHNBRLOG}"
   exit ${SQL_EXIT_STATUS}
fi

#######
####### Add batch number to list of parameters #################################
#######
ParamList="${ParamList} -param BatchNumber=${BatchNumber}"

NOW=`date`
echo
echo ${NOW} ${PROG} Parameter list: ${ParamList}
echo

#######
####### DataStage EXECUTION ####################################################
#######

NOW=`date`
echo "${NOW} ${PROG} Executing DataStage dsjob program..."

echo ${BinFileDirectory}/dsjob -server ${DSSERVER} -user ${DSUSERID} -password ${DSPASSWORD} -run -wait ${ParamList} ${DATASTAGEPROJECT} ${JobName} 2>&1 > ${LogFileName}
echo
echo "${BinFileDirectory}/dsjob -server ${DSSERVER} -user ${DSUSERID} -password ${DSPASSWORD} -run -wait ${ParamList} ${DATASTAGEPROJECT} ${JobName} 2>&1 > ${LogFileName}"
echo

eval ${BinFileDirectory}/dsjob -server ${DSSERVER} -user ${DSUSERID} -password ${DSPASSWORD} -run -wait ${ParamList} ${DATASTAGEPROJECT} ${JobName} 2>&1 >> ${LogFileName}

jobwaiting=`grep "Waiting for job..." ${LogFileName}`
if [ "${jobwaiting}" != "Waiting for job..." ]; then
   NOW=`date`
   echo ${NOW} ${PROG} "DataStage failed to start the job"
   failedstart=1
else
   NOW=`date`
   echo ${NOW} ${PROG} "DataStage successfully started the job"
   failedstart=0
fi
NOW=`date`
echo ${NOW} ${PROG} "Retrieving job information"

${BinFileDirectory}/dsjob -server ${DSSERVER} -user ${DSUSERID} -password ${DSPASSWORD} -jobinfo ${DATASTAGEPROJECT} ${JobName} >> ${LogFileName}

#######
####### CHECK STATUS ###########################################################
#######

ERROR=`grep "Job Status" ${LogFileName}`
ERROR=${ERROR##*\(}
ERROR=${ERROR%%\)*}

if [ "${failedstart}" != 0 ]; then
   NOW=`date`
   echo ${NOW} ${PROG} "The job failed to start"
   AuditStatus="FAILURE"
   Comments="MasterControl aborted"
   EXIT_STATUS=1
else
   if [ "${ERROR}" = 1 -o "${ERROR}" = 2 ]; then
      NOW=`date`
      echo ${NOW} ${PROG} "The job completed successfully"
      AuditStatus="SUCCESS"
      Comments=""
      EXIT_STATUS=0
   else
      NOW=`date`
      echo ${NOW} ${PROG} "The job aborted"
      AuditStatus="FAILURE"
      Comments="MasterControl aborted"
      EXIT_STATUS=1
   fi
fi

FailedJobCount=`grep -i FAILED ${LogFileDirectory}/${JobName}.log|wc -l|cut -b1-9`
FailedJobCount=`expr ${FailedJobCount} + 0`
echo ${NOW} ${PROG} The number of failed jobs is [${FailedJobCount}]

if [ "${FailedJobCount}" != 0 ]; then
   NOW=`date`
   echo ${NOW} ${PROG} "The job had failed processes"
   AuditStatus="FAILURE"
   Comments="MasterControl had ${FailedJobCount} failed processes"
   EXIT_STATUS=1
fi
StoppedJobStreamCount=`grep "JOB STREAM STOPPED" ${LogFileDirectory}/${JobName}.his|wc -l|cut -b1-9`
StoppedJobStreamCount=`expr ${StoppedJobStreamCount} + 0`
if [ "${StoppedJobStreamCount}" != 0 ]; then
   NOW=`date`
   echo ${NOW} ${PROG} "The job stream was STOPped or KILLed"
   AuditStatus="FAILURE"
   Comments="MasterControl job stream was STOPped or KILLed"
   EXIT_STATUS=1
fi

#######
####### AUDIT ##################################################################
#######

echo
echo "${NOW} ${PROG} About to update ETL_BATCH_AUDIT with status information..."
EndTimestamp=`date '+%Y-%m-%d %H:%M:%S'`

SQLstring="UPDATE ETL_BATCH_AUDIT A \
              SET A.END_TIMESTAMP = TO_DATE('${EndTimestamp}','YYYY-MM-DD HH24:MI:SS'), \
                         A.STATUS = '${AuditStatus}', \
                       A.COMMENTS = '${Comments}', \
                A.RUNTIMESETTINGS = '${ParamList}' \
            WHERE (A.BATCH_NBR = ${BatchNumber});"

NOW=`date`
echo ${NOW} ${PROG} Audit SQL ${SQLstring}
SQLScriptFileName=${TempFileDirectory}/${PROG}_${JobName}_end.sql
echo ${SQLstring} > ${SQLScriptFileName}

${CommonScriptFileDirectory}/comSQLPlus.ksh ${ParameterFile} IRDSN IRUserID ${SQLScriptFileName}
SQL_EXIT_STATUS=$?
if [ "${SQL_EXIT_STATUS}" != 0 ]; then
   NOW=`date`
   echo ${NOW} ${PROG} Failure to connect/update into ETL_Batch_Audit table!
   exit ${SQL_EXIT_STATUS}
fi

#######
####### EXIT ###################################################################
#######

NOW=`date`
echo ${NOW} ${PROG} Complete, exiting with status [${EXIT_STATUS}]
exit ${EXIT_STATUS}


好东西，先收着，再研究。

在DataStage中job不能删除，如果试着编译又出现如下错误：
Error calling subroutine: DSR_EXECJOB (Action=1); check DataStage is set up correctly in project ETUDES
(Subroutine failed to complete successfully (30107))

+

(40503) A call to an OLE server has failed, or a runtime error occured within the ole itself.

解决办法：
1 首先运行命令：DELETE FROM DS_JOBS WHERE NAME = 'JobName'
2 再运行DS.CHECKER ，注意要运行DS.CHECKER必须要有exclusive access到project
，通过如下命令可以看到有谁还连接到了服务器：netstat -a |grep dsrpc

                                                            一 BASIC介绍

1 一般的BASIC程序是如下一个格式
[ label ] statement [ ; statement … ] <Return>
2 关于subroutine
a 如果嵌入到了程序中用GOSUB调用，如果在外部，使用CALL
b 使用return返回到主程序，如果在return之前subroutine结束了，自动return
c 如果subroutine中有ABORT STOP CHAIN则永远回不到主程序。
3 注释
以REM * ! $^* 开头的为注释
4 创建一个BASIC 程序的步骤
使用CREATE.FILE命令创建一个type 1或type 19 DataStage file来保存BASIC程序的源码。(在DSEngine目录下运行CREATE.FILE命令，它将提示你输入文件名等参数)

使用DataStage Editor或其他编辑器创建BASIC程序源码，这个源码成为你刚才创建的文件的一条记录。

使用BASIC命令来编译你程序，这将创建一个包含目标代码的文件。目标文件的名字不重要，因为总是通过源文件名来引用程序。

可以用RUN命令来执行你的程序，使用RAID命令来调试你的程序。

                                                                二、基础知识
数据类型、变量和操作符
所有数据在内部都是以字符串存储，在运行时转换为具体类型。
主要有三种数据类型：字符串、数字型、和unknown（null）

字符串
在BASIC源码中，字符串产量是ASCII字符的序列，所以不能有0（null）和10（linefeed）字符，而是要使用CHAR(0)和CHAR(10)
数字型
范围从10(-307)到10(307)，最多15位小数
unknown
null在内部是由单一字节CHAR(128)代表的，是一个string，可以用系统变量@NULL.STR来引用它。在BASIC程序中可以使用系统变量@NULL来引用null值。
一个字符串与null连接结果为null

数组
dimensioned array
在向量（一维数组）中，第一个元素的index为1
向量可以看做第二位为1的二维数组（矩阵）。
矩阵和向量的第一个元素分别用vector.name(0)和matrix.name(0,0)表示。它可以在MATPARSE、MATREAD和MATWRITE语句中用来存储不合格的字段。

动态数组
动态数组的最高层是字段，由(F)分割。字段包含有(V)分割的值组成。每个值又可以包含由(S)分割的subvalue。

字符串的比较
字符串的比较是一个个的比较每个字符的ASCII的值。如果两个字符串可以转换为数字，总是比较两个数字的数值大小。任何值与null相比较的结果都是unknown。空串不是简单的等于0。空格看做小于0。

字符串匹配
X代表任何字符 A代表所有字母 N代表所有数字 前面的数字代表重复次数。如果前面的数字为0，代表后面的字符重复0到N次的任意次数都可以。

0是false 空串是false null非true非false

锁 事务和隔离级别
锁
锁有两种粒度：细粒度（记录锁定）和粗粒度（文件锁定）
DataStage支持以下几种锁：
记录共享锁
记录更新锁
文件共享锁
文件意向（intent）锁 介于共享与排他之间
文件排他锁

事务
事务可以嵌套
子事务在父事务提交或回滚之前开始的话，子事务将变成活动事务，而父事务将继续存在但是是非活动的。当子事务提交或回滚之后，父事务将再次变为活动的。
在最高层的事务提交之前，将缓存所有子事务的数据库操作。当进行一个读取操作时，首先会去读取缓存的数据，如果找不到才会从数据库文件中读取。

论坛上见到的，记录一下，以备後用
Process p = Runtime.getRuntime().exec( MProperties.DS_EXE_PATH + "/dsjob -run -wait -param lowDate=" + lowDate + " -param highDate=" + highDatePlusOne + " -param dsOutDir=" + MProperties.DS_OUT_DIR + " -param IDB_user=" +MProperties.IDB_USER + " -param IDB_pass=" +
MProperties.IDB_PASS + " " + MProperties.DS_PROJECT + " " + MProperties.DS_JOB);
p.waitFor();
return p.exitValue() == 0;

另外如果有RTI，更好的方法如下：
The best way for this to be accomplished is by using the RTI (Real Time Integration) capabilities that we added in 7.x. This allows you to fully expose a DataStage job as a service, to be then invoked via several different mechanisms, depending on your needs....

.....the job can be exposed as a classic web service, using SOAP over HTTP...we generate and publish the WSDL for this within RTI...

....the job can be exposed via our EJB binding and called appropriately

....the job can be exposed for contact via JMS, and either passed a text message or SOAP envelope....

Not only can you simply pass job parameters via this architecture, you can actually have the entire transformation job itself (columns on the ultimate input and output links) passed and returned to your calling client.

Additionally, the entire job control API, which you may already be familiar with in its API or command line implementations, has been exposed as a set of methods available via Web Services.
主要就是通过将job包装为web service或EJB或JMS来调用，可以传参数，还可以将整个job本身（最终的输入和输出链接）作为参数传递或者返回到调用的客户端。

通过右键添加link
鼠标右键点击起始stage，按住右键移动鼠标到目标stage。还有一种方法就是把鼠标放在起始stage的边缘等到变为一个小圆圈里面有一个叉的时候拖动鼠标到目标stage。

DataStage中默认和隐式类型转换时注意的问题
当从源向目标映射数据时，如果类型不一致，对于有些类型我们需要在modify或transfomer stage中通过函数进行转换，对于有些系统会自动完成类型转换，在类型转换过程中，注意以下几点：
1 在变长到固定长度字符串的默认转换中，parallel jobs用空格（ASCII 20）字符来填充剩余长度。联系到前面做的一个parallel job，当目标是变长时（当时源也是变长，但是好像源的精度要小些），但是字符串实际的长度没有指定的变长那么长的话，系统会自动用空格（ASCII 20）来填充（具体在哪个stage填充的不知），而且环境变量APT_STRING_PADCHAR的默认值也是空格（ASCII 20）。
2 通过环境变量APT_STRING_PADCHAR可以改变默认的填充字符null（ASCII 0），同时一般还要加上trim就能解决空格问题，此问题非常普遍。
3 有个PadString 函数可以用来用指定的字符来填充一个变长的字符串到指定的长度。这个函数的参数不能使固定长度字符串，如果是固定长度的先转化为变长。
 
Copy Stage作为占位符
在job开发中，当你暂时不知道该使用哪个stage时，可以使用copy stage作为占位符，注意不要把Force属性设为True，在不把Force设为True时，复制将在运行时进行优化。

使用Schema file来指定meta data
1 可以通过列定义和schema file两种方式来指定meta data，值得注意的一点是Note that, if you use a schema file, you should ensure that runtime column propagation is turned on. Otherwise the column definitions specified in the stage editor will always override any schema file.即，如果想通过schema file来指定meta data，必须勾上runtime column propagation，否则总是使用列定义来作为meta data。
2 If you are using a schema file on an NLS system, the schema file needs to be in UTF-8 format.如果你的datastage使用了NLS system，那么schema file必须是UTF-8格式的。文档中说可以使用Sequential stage来完成转码。
 
通过Server Shared Container在Parallel Job中添加Server Job Stage
You create a new shared container in the DataStage Designer, add Server job stages as required, and then add the Server Shared Container to your Parallel job and connect it to the Parallel stages.在Designer中创建一个server shared container，添加需要的Server job stage，再把server shared container添加到parallel job并连接到其他parallel stage。
 
几个环境变量的具体路径
%SERVERENGINE% – DataStage engine account directory(normally C:\Ascential\DataStage\Engine on Windows and <dsadminhome>\Ascential\DataStage\DSEngine on UNIX).
%PROJECT% – Current project directory.
%SYSTEM% – System directory on Windows NT or /usr/lib on UNIX.

一句不太懂的话，先做记录
If NLS is enabled, avoid using the built-in Iconv and Oconv functions to map data unless you fully understand the consequences of your actions.在使用了NLS时，尽量避免使用Iconv and Oconv来映射日期，因为可能产生一些意想不到的结果。

I installed Chinese version of Eclipse.When I want to switch to English version,I use -nl en_us option and everything is just ok.

One day,I installed tortoise svn plug-in.In order to configure the svn plug-in,I first use -nl en_us option to start my Eclipse (just as I often do),then I open preference,hoping to see the tortoise in the preference and configure it.The problem is that I can not see tortoise in the preference as supposed.After trying many means to figure out the problem,I get the right answer:do not use -nl en_us option to start Eclipse and you can see tortoise in the preference.You need only to do this once,then you can use -nl en_us option to start your Eclipse as you usually do and you can still see tortoise in the preference.

     现在你在各大网站、论坛、博客都能看到SOA的身影，有很多人都在怀疑SOA是不是另一个buzzword。不过J-SOFA 对这种看法做出了一些回击，它给出了一些很实际的东西让我们感受SOA的力量。你可以去http://www5f.biglobe.ne.jp/~webtest/jsofa/具体了解一下。
下面是它的首页对它的介绍：
J-SOFA是编排服务组件的一个轻量级框架，这些服务组件包括JSF中的Backing Bean 和Struts中的Action，等等。它使得我们可以把业务流程从动作中分离出来并且它还帮助我们将业务逻辑实现为服务。这让你的应用成为实现了SOA的应用。