FluentFTP
FluentFTP copied to clipboard
robinrodricks
Access FTP Responses on data transfer connection during DownloadFile and UploadFile
FTP OS: Windows
FTP Server: IBM FTP (z/OS)
Computer OS: Client or server? z/OS for server.
FluentFTP Version: 34.0.0
I got an error when trying to upload a file. It ran out of space. However I cannot find a way to see what the server response is which contains the details of the problem. I can see it when I am in debug mode in the Immediate Window in Visual Studio, but I cannot seem to find how to get at that from my program.
Another reason I need this is that I submit jobs to run on the mainframe using FTP. I need to access the response in order to get the job number assigned so I can download the output of the job from the output queue.
Logs : Don't know how to do logs. Another issue with finding stuff in the documentation. I can see some things regarding logs, but have no idea how to set it up and produce them. VB.NET example would help.
Additional note With zOSFTP, you can run a job by doing a SITE command telling JES to submit a job when you do a GET for the file that contains the JCL to run the job. What it returns is the output of that job, not the JCL. When I do the Get, this is what I see in the Immediate Window in Visual Studio: Command: RETR cmpa.pvt.source(jlistcat) Response: 125-Submitting job cmpa.pvt.source(jlistcat) FIXrecfm 80 Response: 125 Unable to send CMPA.CMPA.PVT.SOURCE(JLISTCAT) Status: Disposing FtpSocketStream... Response: 550 Transfer aborted What I get in the FTPReply object is just that last line: Code:550 ErrorMessage:Transfer aborted InfoMessages: Message:Transfer aborted Success:False Type:PermanentNegativeCompletion
I'd like that too, this is preventing us from accessing the error code from a production log.
I found a way for myself to get results from CLI, which works for my use-case; I just enable this :
FtpTrace.LogToConsole = true;
A bit verbose on the CLI but it works for us.
Mine is not a CLI app, and the purpose will be to interpret the raw responses into something the user will understand who knows nothing about FTP. So I need to get at it programmatically.
Firstly, when any command fails it will throw an FtpException and the details will be in that object .
About logs, example code is there in the examples folder for vb and c#. Unless you want to access logs without it going to CLI - even that is possible just read the wiki.
On Thu, Aug 19, 2021, 3:44 AM MihaIvan @.***> wrote:
Mine is not a CLI app, and the purpose will be to interpret the raw responses into something the user will understand who knows nothing about FTP. So I need to get at it programmatically.
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/robinrodricks/FluentFTP/issues/747#issuecomment-901464375, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABT3UKV5ECYNAC46NSWY4GTT5QWC3ANCNFSM5B4NTEMA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email .
https://github.com/robinrodricks/FluentFTP/wiki/Logging
On Thu, Aug 19, 2021, 8:54 AM Robin Rodricks @.***> wrote:
Firstly, when any command fails it will throw an FtpException and the details will be in that object .
About logs, example code is there in the examples folder for vb and c#. Unless you want to access logs without it going to CLI - even that is possible just read the wiki.
On Thu, Aug 19, 2021, 3:44 AM MihaIvan @.***> wrote:
Mine is not a CLI app, and the purpose will be to interpret the raw responses into something the user will understand who knows nothing about FTP. So I need to get at it programmatically.
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/robinrodricks/FluentFTP/issues/747#issuecomment-901464375, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABT3UKV5ECYNAC46NSWY4GTT5QWC3ANCNFSM5B4NTEMA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email .
Here is the test code snippet: Dim wkStream As New MemoryStream Dim wkJCL As String = "cmpa.pvt.source(jlistcat)" Dim wkLine As String = "" Dim wkChar As String Dim wkBYte As Byte
'set the transfer type to ASCII so we can actually use the data
_mFTP.DownloadDataType = FtpDataType.ASCII
Try
mFTP.Download(wkStream, wkJCL)
Catch ex As Exception
Dim wkMsg As String = ex.Message
If IsNothing(ex.InnerException) Then
Else
wkMsg += vbCrLf & ex.InnerException.Message
End If
MsgBox("Run error: " & wkMsg)
End Try
ShowReply("Run job", mFTP.LastReply)_
I just reran my test, and the Download method did NOT throw an exception. The information in the LastReply object was: Code:550 ErrorMessage:Transfer aborted InfoMessages: Message:Transfer aborted Success:False Type:PermanentNegativeCompletion
Which is nice to know, but not useful. Here is what I see in the Immediate Window in Visual Studio with line numbers added for reference:
- Command: RETR cmpa.pvt.source(jlistcat)
- Response: 125-Submitting job cmpa.pvt.source(jlistcat) FIXrecfm 80
- Response: 125 Unable to send CMPA.CMPA.PVT.SOURCE(JLISTCAT)
- Status: Disposing FtpSocketStream...
- Response: 550 Transfer aborted
The LastReply only shows response 5. The real information I need is from line 2 and 3. It appears that 125 may not exactly be an error all the time, since line 2 seems to be informational. However line 3 is the one I really need to see.
I will take a look at the examples again, but I just scanned the list, and there is nothing specific in the description that tells me that the example is about logs. I will dig deeper though. I usually RTFM, but not every line of it. And index would be helpful to look for things not in the descriptions.
@MihaIvan this is the index - https://github.com/robinrodricks/FluentFTP/wiki
There is also API-wise index but may not be 100% up to date.
I get your use case, let me see what I can do.
@MihaIvan Did you check the InfoMessages property of the LastReply object? It should have the response lines...
If its not in InfoMessages it could be because your server is sending those responses on the data transfer connection.
Added to the bucket list. We will pick this up as and when we have free time. Comment on this issue if you want us to prioritize it. Thanks!
@MihaIvan @fdrobidoux
I know this has been sitting there for a while. Could I ask the participants if this is still required by them?
The following assumes a previously issued SITE FILETYPE=JES command to have been executed.
I can confirm that the following sequence:
Command: RETR TEST
Response: 125-Submitting job TEST FIXrecfm 80
Response: 125 Unable to send ASTYS.TEST
Status: Disposing FtpSocketStream...
Response: 550 Transfer aborted
Status: Failed to download file.
does not include the two 125 messages to the lastReply object of the FTP client.
Why not? I am working on that.
But I have identified a small usage issue, I think.
You wrote:
With zOSFTP, you can run a job by doing a SITE command telling JES to submit a job when you do a GET for the file that contains the JCL to run the job. What it returns is the output of that job, not the JCL.
That is simply not true.
When SITE FILETYPE=JES is active, a
STOR filename will upload a jcl file from your client system to the host and submit this as a job.
LIST will list the job output queue, from which you can select a job output to download to your client system
RETR jobname.n will then download the jobs output file number n.
Here is an example:
FTP_Sess.Config.ListingDataType = FtpDataType.ASCII;
FTP_Sess.Config.UploadDataType = FtpDataType.ASCII;
FTP_Sess.Execute("SITE FILETYPE=JES");
FTP_Sess.UploadFile("D:\\temp\\JES\\LISTCAT", "LISTCAT", FtpRemoteExists.NoCheck, false, FtpVerify.None);
assuming you have a file on your system with some jcl. This upload will transfer the jcl from the file to the JES2 internal reader and submit the job.
The LastReply object after the upload will look like this:
You see, in the InfoMessages, there is 250-It is known to JES as JOBnnnnn.
So then you do:
FTP_Sess.Config.ListingDataType = FtpDataType.ASCII;
FTP_Sess.Config.DownloadDataType = FtpDataType.ASCII;
FtpStatus Jest = FTP_Sess.DownloadFile("D:\\temp\\JES\\LISTCAT.OUT.1", "JOB00061.1", FtpLocalExists.Overwrite, FtpVerify.None);
Works like a charm.
Here is the complete log of the process:
Command: SITE FILETYPE=JES
Status: Waiting for response to: SITE FILETYPE=JES
Response: 200 SITE command was accepted
> UploadFile("D:\temp\JES\LISTCAT", "LISTCAT", NoCheck, False, None)
> OpenWrite("LISTCAT", ASCII)
> OpenPassiveDataStream(AutoPassive, "STOR LISTCAT", 0)
Command: EPSV
Status: Waiting for response to: EPSV
Response: 229 Entering Extended Passive Mode (|||1913|)
Status: Connecting to ***:1913
Command: STOR LISTCAT
Status: Waiting for response to: STOR LISTCAT
Response: 125 Sending Job to JES internal reader FIXrecfm 80
Status: Disposing FtpSocketStream...
Status: Waiting for a response
Response: 250-It is known to JES as JOB00060
Response: 250 Transfer completed successfully.
> DownloadFile("D:\temp\JES\LISTCAT.OUT.1", "JOB00061.1", Overwrite, None)
> OpenRead("JOB00061.1", ASCII, 0, 0)
Command: TYPE A
Status: Waiting for response to: TYPE A
Response: 200 Representation type is Ascii NonPrint
> OpenPassiveDataStream(AutoPassive, "RETR JOB00061.1", 0)
Command: EPSV
Status: Waiting for response to: EPSV
Response: 229 Entering Extended Passive Mode (|||1915|)
Status: Connecting to ***:1915
Command: RETR JOB00060.1
Status: Waiting for response to: RETR JOB00061.1
Response: 125 Sending data set BFSYS.BFSYSA.JOB00061.D0000002.JESMSGLG
Status: Disposing FtpSocketStream...
Status: Waiting for a response
Response: 250 Transfer completed successfully.
You can also do:
FTP_Sess.Config.ListingDataType = FtpDataType.ASCII;
FtpListItem[] jesList = FTP_Sess.GetListing("LIST", FtpListOption.ForceList | FtpListOption.NoPath | FtpListOption.NoImage);
var listLine = jesList[0].Input;
which would give you in listLine: "BFSYSA JOB00069 BFSYS OUTPUT A RC=0004 4 spool files ". Parsing that will tell you there are 4 files, JOB00061.1 through JOB00061.4 that you could retrieve.
I can confirm that the following sequence:
Command: RETR TEST
Response: 125-Submitting job TEST FIXrecfm 80
Response: 125 Unable to send ASTYS.TEST
Status: Disposing FtpSocketStream...
Response: 550 Transfer aborted
Status: Failed to download file.
does not include the two 125 messages to the lastReply object of the FTP client.
Why not? I am working on that.
After some debugging with my personal V42 (A new GetReply(...) version), which goes into a bit more detail in its logging: I can see the following:
Command: RETR JfB00060.1
Status: Waiting for response to: RETR JfB00060.1
Response: 125-Submitting job JfB00060.1 FIXrecfm 80
Response: 125 Invalid data set name "JfB00060.1". Use MVS Dsname conventions.
Status: Disposing FtpSocketStream...
Status: Waiting for a response
Response: 550 Transfer aborted
As you can see, the 125-**** messages are the direct response to the RETR command, similar to when it answers with 125 Sending data set .... This is caught by GetReply(...) correctly.
BUT
As you can also see, the next response, in this case the 550 Transfer aborted denotes the end of the transfer. Sometimes successful, in this case not. It replaces the LastReply object and in effect hides the 125-*** messages, which were the previous replies.
This explains why these responses have "dissappeared" and it is not because they are sent on the data channel.
I will work on a possible solution to this.
When this PR #988 is merged, you can programatically access the intermediate replies on your process.