Author Topic: empty clientraw on primary ftp webserver when second ftp running (Read 15300 times)

choc-a-holic · « **on:** July 21, 2016, 08:21:29 PM »

Recently we needed to create a second ftp task to send files to a different server, however this causes intermittent empty clientraw files to appear om the primary ftp site. When I disable the second ftp task the problem goes away.

This problem seems to occur in the evening mainly and can last from minutes to hours. We have seen it during the day from time to time.

I downloaded & upgraded to the very latest version of WD today however the problem is still there.

I have been looking at the ftp log and compared the uploads when it working fine:

Uploading file C:\wdisplay\webfiles\clientraw.txttemp -> / clientraw.txttmp
PASV
227 Entering Passive Mode (I removed for security reasons)
STOR clientraw.txttmp
150 Accepted data connection
226-File successfully transferred
226 0.008 seconds (measured here), 95.08 Kbytes per second
File uploaded at time :19:0:38
RNFR clientraw.txttmp
350 RNFR accepted - file exists, ready for destination
RNTO clientraw.txt
250 File successfully renamed or moved

and when it is not working ….this is what I have found - see txt in green as it may be a clue to what is happening:

Uploading file C:\wdisplay\webfiles\clientraw.txttemp -> / clientraw.txttmp
PASV
503 Need RNFR before RNTO
141: 503 Need RNFR before RNTO
File uploaded at time :19:32:51
RNFR clientraw.txttmp
227 Entering Passive Mode (I removed for security reasons)
RNTO clientraw.txt
550 Sorry, but that file doesn't exist
141: 550 Sorry, but that file doesn't exist

Hopefully we can find a solution to this problem

gabba · « **Reply #1 on:** July 21, 2016, 09:38:27 PM »

Hi

The server is saying it needs to see a rename from command before a rename to command.

Directory Commands and Deleting Files: MKD , RMD , DELE , RNFR , and RNTO commands

It looks like your ftp server supports the commands. The question is why doesn't it support it consistently. As its odd behaviour on your hosts side I'm afraid it's back to them for some ideas. Sorry!

Alternatively You might be able to set up a seperate ftp program to send the file up to the second server although that seems a bit messy and excessive

choc-a-holic · « **Reply #2 on:** July 22, 2016, 03:43:20 PM »

I would agree with you if the problem happened when I only had the one ftp upload enabled, however this problem only occurs when I have two ftp uploads enabled...I was wondering if somehow the two ftp job commands were getting muddled up with wrong command going to the wrong server. I cannot think of anything else that would cause this type of issue.

gabba · « **Reply #3 on:** July 22, 2016, 05:50:32 PM »

Here's my logic

It works most of the time for you. If it didn't work at all then I would suspect the software or at least how it was set up.

However having it work most of the time kinda rules that out especially if you aren't changing anything on your side.

Of course switching the file rename switch off may help but the server that is working works best with the rename command and I don't think there is an option to have one connection rename the file and the other one not.

I did wonder if the rnfr command was getting lost or dropped somewhere. That's a possibility I guess but figuring out what is causing that might be a challenge. Can you remind me and others who might be able to help if it happens at similar times during the day and what type of internet connection you are on?

choc-a-holic · « **Reply #4 on:** July 22, 2016, 06:11:09 PM »

Unfortunately we are only on ADSL1....the joys of being at the furthermost point from an old exchange. The problem occurs mainly in the evenings but at times it happens during the day. Our home internet connection is hammered all day and gets limited use in the evenings and the computer with WD gets used a lot during the day.

It has just been upgraded to Window 10 today...I was running out of time to get the free upgrade.

The rename option works fine when I just have one ftp enabled....doesn't miss a beat.

Weather Display · « **Reply #5 on:** July 27, 2016, 07:24:45 AM »

when you say you have 2 uploads enabled
do you mean you have set to use the 2nd real time ftp...and is that then being uploaded to a different ftp server?
(and why?)

choc-a-holic · « **Reply #6 on:** July 27, 2016, 09:49:29 AM »

Yes I have the second ftp server setup to ftp the clientraw.txt file for this website. It was the only way we could get things working for this website.

Weather Display · « **Reply #7 on:** July 27, 2016, 12:33:36 PM »

when you say this web site
I think you are meaning to get data to localweather regional meso data?
and so that is a different FTP server than your normal FTP server?
(a bit vague re the information that you have provided so I am just guessing)

choc-a-holic · « **Reply #8 on:** July 27, 2016, 01:00:38 PM »

Sorry...the live NZ map on this website(localweather.net.nz) was reading my clientraw.txt file from my web hosting company, however they were finding they could no longer read the file with the software they are using. I setup a 2nd realtime ftp to ftp the file to an alternative location provided by localweather.net.nz and now everything is working for them.

Unfortunately that has meant that I am experiencing problems with the clientraw.txt on my own website which uses the main realtime ftp. If I disable the 2nd realtime ftp, my problem is fixed.

Weather Display · « **Reply #9 on:** July 27, 2016, 01:33:03 PM »

so its the normal ftp, using the clientrawrealtimeftp.exe, to your web site, where the problem is
instead of too the forum server?
you could try the latest version of the clientrawrealtimeftp
http://www.weather-display.com/downloadfiles/clientrawrealtimeftp.zip

gabba · « **Reply #10 on:** July 27, 2016, 07:39:08 PM »

Choc-a-holic needs to upload to two servers currently

(a) The first server (mine) works fine. ie Ken Trues meso map reads the clientraw from that my server fine. It needs the rename temporary file function to be in place for it to work best. If we switch that off, choc-a-holic gets errors

(b) the second server has choc-a-holic's main website on it and it needs the clientraw on it to show data. A while ago we identified that when the file gets to the second server it has a "hidden" character in it that would otherwise cause the meso script to not read the file correctly. The problem is in the way that Kens script handles the hidden character, as Choc-a-holics website seems to manage it fine. So that means we cant use the file on that server for the meso map.

So most of the time it works ftping to the two servers, except some of the time it doesn't

When it doesn't it looks like it gets stuck on the rename function. Since it works most of the time, but not all the time, I'm guessing it is something to do with the connection, or the ftp server on the host side (shared so it might be busy), not the FTP program in WD. I'm guessing it uploads the client raw file as the tmp file, then tries to create the clientraw.txt and copy the tmp file to that, can't do that, so it ends up with a blank file

What might help is the ability to use the rename function on one server, but not on the other. Currently I think WD only supports the rename on both servers or no rename on both servers. - unless I'm missing something!

I've played with this for a long time, so my memory is getting sketchy. Choc-a-holic, did we try pointing your website at the clientraw file on my server?

Weather Display · « **Reply #11 on:** July 27, 2016, 08:51:54 PM »

some servers are known to add a special control character to certain files (at the start of the file), which can happen to the clientraw file...that is a server side problem though

as the ability to have the 2nd real time clientraw have its own rename option, I could add that..

but first see how the latest version works (see link above)

choc-a-holic · « **Reply #12 on:** July 27, 2016, 09:53:43 PM »

Gabba - We have not tried pointing my webserver to the file on your webserver....but then I have quite a bit happening on my server with WDlive, steel gauges, records, graphs & webcam.

The problem occurs whether we have the rename function turned on or off. We just get more info in the ftp log when rename is turned on. The problem does not occur when I remove the second ftp server which makes it hard to point the issue at my website provider.

I will load the new clientrawrealtimeftp tomorrow and see how it goes. I really do hope we can find a solution to this.

It could be a link issue which we will never be able to solve

- it is getting very frustrating as we use the file on our web server all the time to update our android phone apps and of course when we look in the evenings its just not working and often in the early hours of the morning as well when my partner wants to know how cold it is outside or how much rain we got overnight. Our neighbors all look at the website too....it is just being so unreliable lately. If we cannot solve soon, I am afraid I will have to give up on the 2nd ftp server for the map.

JennyLeez · « **Reply #13 on:** July 28, 2016, 12:33:19 AM »

Before you give up, a very easy solution to this which has been mentioned somewhere by memory is to run one clientraw.txt upload to your server by WD.
Use a separate FTP piece of software to upload the 2nd one to this forum.

Cumulus only allows the realtime.txt to be uploaded once.
I needed it on 2 servers so the above is what I am doing.
Cumulus uploads to one server. My FTP software uploads to the other. (Plus a few other files)

I can send you the software I am using. Very easy to set up.

It is just a shame your own server is screwing up the clientraw.txt as there is the root of the problem.

Cheers
Jenny

TonyC · « **Reply #14 on:** July 28, 2016, 08:35:20 AM »

Just my two bobs worth.
Don't know what frequency you want to ftp the file, but I have a number of batch files which ftp files to various servers. I use Windows Scheduler to launch them. I think every 15 minutes is the highest frequency Scheduler will run progs.

cheers

Tony C.

Weather Display · « **Reply #15 on:** July 28, 2016, 11:31:58 AM »

you can set to have the clientrawrealtimeftp restart instead of re login..in the webfiles/web page setup .real time ftp settings in WD
e.g and set that to very 5 minutes
as a restart instead of re login might help the situation?
(i.e when the problem occurs, what gets it going again ?)

choc-a-holic · « **Reply #16 on:** July 28, 2016, 04:24:08 PM »

I have just installed the latest ftp files provided by Brian

Quote

you can set to have the clientrawrealtimeftp restart instead of re login..in the webfiles/web page setup .real time ftp settings in WD
e.g and set that to very 5 minutes
as a restart instead of re login might help the situation?

I have changed the ftp server to restart every 5 minutes and we will see what happens.

Quote

when the problem occurs, what gets it going again ?

If I delete the clientraw.txt file then the problem is resolved for a period of time, however I usually just leave it and it comes right by itself.

choc-a-holic · « **Reply #17 on:** July 28, 2016, 04:30:13 PM »

I just noticed the versions of ftp used are different, should they be different? - see attachment

Weather Display · « **Reply #18 on:** July 28, 2016, 11:12:15 PM »

yes...as they are 2 different programs
I will be updating though the 2nd real time ftp version...to add support for secure ftp

also I could add in that when an error occurs with the file rename, then the file is deleted...

Weather Display · « **Reply #19 on:** July 31, 2016, 06:38:50 AM »

I have an updated 2nd real time ftp version available now:
http://www.weather-display.com/downloadfiles/realtimeftp2.zip

News:

Author Topic: empty clientraw on primary ftp webserver when second ftp running (Read 15300 times)