sql server - SSIS DelayValidation randomly not working (Flatfile Connection Manager with Expression) -


have ever had weird behavior in ssis 2008, delayvalidation on flatfile connection manager randomly not working?

here scenario struggeling on ssis 2008 (non r2):

  • my package covers import of 30 different *.csv flat files database

  • csv files delivered each day dynamically created folder (d:\datadelivery\deliveryid_<generateddeliveryid>)
    (e.g. "d:\datadelivery\deliveryid_73\", "d:\datadelivery\deliveryid_74\", ..)
    ==> here csv files imported via ssis

  • the delivery id current import determined stored proc @ beginning of ssis package (stored ssis variable "ideliveryid")

  • based on determined delivery id build complete import path using variable called "simportpath" expression (evaluateasexpression=true):
    "d:\datadelivery\deliveryid_"+ (dt_wstr, 1252)@[user::ideliveryid] +"\"

..

  • the generated import path used in expression connection property of each flatfile connection manager, e.g.:

    • @[user::simportpath] + "filea.csv"
      (expression connection property in connection manager a)
    • @[user::simportpath] + "fileb.csv"
      (expression connection property in connection manager b)
    • @[user::simportpath] + "filec.csv"
      (expression connection property in connection manager c)

  • the delayvalidation property of each connection manager set true!!!

  • so delivery id 73 connection properties of flatfile connection managers
    should @ runtime resolve to:
    d:\datadelivery\deliveryid_73\filea.csv
    d:\datadelivery\deliveryid_73\fileb.csv
    d:\datadelivery\deliveryid_73\filec.csv

but happens right in reality:

  • when running in visual studio runs fine intended
  • when running in production via sql server agent, following happens:
    • sometimes package runs fine (all files imported)
    • sometimes filea imported correctly while in same process fileb stops message "cannot open datafile d:\datadelivery\deliveryid_0\fileb.csv"
      (error in pre-execute phase)
      => see delivery id 0 here => expression not resolving
    • sometimes fileb or filec imported correctly while in same process filea stops message "cannot open datafile d:\datadelivery\deliveryid_0\filea.csv"
      (error in pre-execute phase) => see same error

==> in cases generated path (variable simportpath) resolves correctly (i logging this)
==> seems delayvalidation randomly working of connection managers while not working others ==> tried set delayvalidation=true whole package single tasks , containers ..still same behaviour have ever experienced such behaviour? if yes, how did deal this?

thanks lot suggestions.
jens


Comments

Post a Comment

Popular posts from this blog

c++ - End of file on pipe magic during open -

i have c++ application in starting process(wireshark) following. if (fp == null){ fp = popen(processpath, "r"); //processpath process want start if (!fp){ throw std::invalid_argument("cannot start process"); } fprintf(fp, d_msg);//d_msg input want provide process } else if(fp != null){ fprintf(fp, d_msg); } the problem when execute c++ application, start wireshark error end of file on pipe magic during open what should avoid that? also tried using mkfifo create named pipe , execute it. used this: if (fp == null){ system("mkfifo /tmp/mine.pcap"); fp = popen("wireshark -k -i /tmp/mine.pcap", "r"); if (!fp){ dout << "cannot start wireshark"<<std::endl; throw std::invalid_argument("cannot start wireshark"); } input = fopen("/tmp/mine.pcap", "wb"); fprintf(input , d_msg); fclose(input)...
Read more

basic authentication with http post params android -

i doing basic authentication passing username , password , using basicnamevaluepair sending post params response service. my method: public stringbuilder callservicehttppost(string username, string password, string type) { // create new httpclient , post header httpclient httpclient = new defaulthttpclient(); httppost httppost = new httppost(webservice + type); httpresponse response = null; stringbuilder total = new stringbuilder(); try { url url = new url(webservice + type); /*string base64encodedcredentials = base64.encodetostring((username + ":" + password).getbytes(), base64.url_safe | base64.no_wrap);*/ string base64encodedcredentials = "basic " + base64.encodetostring( (username + ":" + password).getbytes(), base64.no_wrap); httppost.setheader("auth...
Read more

data.table making a copy of table in R -

i doing this: myfun <- function(inputvar_vec){ # inputvar_vec input vector # # result = output vector return(result) } dt[, result := lapply(.sd, myfun), = byvar, .sdcols = inputvar] i getting following warning: warning message: `in `[.data.table`(df1, , `:=`(prop, lapply(.sd, propeventinlastk)), : invalid .internal.selfref detected , fixed taking copy of whole table, := can add new column reference. @ earlier point, data.table has been copied r (or been created manually using structure() or similar). (and more stuff) .... ` my guess because stacking result vectors (after operation), copy being made? can suggest method remove warning? have done using apply functions , thought should extendable here too. my other question is: can pass chunk of rows data frame (subsetted using statement), , call function myfun operate on that? adding example requested # generate data n = 10000 default=na value = 1 df = data.table(id = sample(1:5000, n, replace=true), ...
Read more