Kaseya Community

getURL only partially downloads file

  • I am using VSA SaaS and have reached my 1GB allowance to upload files to the kServer.

    To get around this I am using OneDrive, sharing out an anonymous link to download my files instead.

    However what I am seeing is the file (zip, MSI or EXE) will only download the first 7k and then stop.

    If I manually access the link on the target endpoint, the file will download successfully.

    Does anyone have any ideas?

  • Are sure it is the first 7k of your download and not a different file that is being downloaded?

    You can check the file by opening it up in Notepad.

    If you see HTML like code the error message that explains the download failure might be in the file, if you rename the extension of the downloaded file  to .htm and open it in a browser you might be able to see the error page.

  • Hi HardKnox,

    When I renamed the file to a htm extension and used Edge to open the file, I was able to download the original file (in this case .zip) as intended.

    Using Notepad just gave some HTML header details but nothing that looks like an error.

    When using the getURL command, the destination filename is simply c:\temp\<name-of-zip-file).

    Am I missing something obvious?

  • After some further investigation, the getURL command is correctly returning the text and HTML contents of the URL.

    I guess my other question is what command can I use to actually download the associated file behind the URL?

  • If you share the file you will have the option to create a link to it that can be provided to those you want to share the file with.  Set it up to allow anyone with the URL access to it w/o login and then copy the URL into your script as the URL to be used by the CURL command.

  • I do not think you're going to be able to do this with OneDrive.

    The link you get when you share a file on OneDrive takes you do a page that has a link to download the file.

    It appears as though the anonymous link from OneDrive is valid to let you open the web page to get the file download link, but the file download link itself can only be accessed from the web page that the anonymous link goes to:

    OneDrive > generate anonymous link (link A)

    Link A takes you to web page that has link to the file you want - Link B

    You cannot access Link B from anywhere except the web page of Link A

    To test, I shared a .zip file from my OneDrive to "anyone with the link can view" setting.

    I then used wget on a Linux VM. (Kaseya uses curl, but wget does the same function)

    When I wget Link A, it downloads a file with html code, as you seem to have gotten.

    However, trying to wget the Link B on a Linux machine, and got error 403: Forbidden.

  • You can do it with OneDrive. Just change the last part of the URL (the part after the ? character) to "download=1". Using a URL that ends with that text will allow the file to be downloaded without being redirected to an HTML page.

  • Thanks for the detailed explanation Eric.

    The link is indeed anonymous and from the sound of it, it involves 2 steps as you have outlined. I thought I was going mad for a while.

    if there is just a way I can get MS edge to open the .htm file which triggers step 2 that would be great.

    Alternatively can anyone recommend a cloud storage solution that will generate a direct download link?

    Instead of using OneDrive, I have tried to use DropBox and Box to no avail.

    Don't really want to have to use a web server and worry about setting up permissions etc if I can avoid it.

  •  That totally worked!  Thanks!

      I just did another wget, using "Link A" , but replacing the end of the url with download=1 as kmarshal suggested and it downloaded the file.

    make sure you specify the downloaded file filename.



    added note to Chris Y
    [edited by: eric at 8:26 AM (GMT -7) on Jul 13, 2020]
  • Hi KMarshall,

    The actual link looks a bit like this:-

    https://1drv.ms/u/s!AlQOwHt0GaCeephK7La7q26tz3A?e=ydO63y

    where would I insert the download-=1 string?

  • After the ? character. The new link should look like this:

    https://1drv.ms/u/s!AlQOwHt0GaCeephK7La7q26tz3A?download=1



    Edited the URL
    [edited by: kmarshall at 8:39 AM (GMT -7) on Jul 13, 2020]
  • I really don't know where I am going wrong.

    Just tried the link and I just get a 7kb file as before.The string reads: https://1drv.ms/u/s!AlQOwHt0GaCeephK7La7q26tz3A?download=1

    Eric you make reference to the filename. I am not sure what you are referring to?



    Amended.
    [edited by: Chris Y at 9:05 AM (GMT -7) on Jul 13, 2020]
  • So...  The problem here is that you can only share files for direct embedding or downloading from your personal OneDrive and not from OneDrive for Business.  That being said, you're also better off using wget over curl.

    To get the file you want via wget (or cURL if you insist) requires that the file not be stored on OneDrive for Business (SharePoint).  It must be in your personal OneDrive (@love.com,@outlook.com,etc..).

    Once the file is where you need it to be in the web interface right-click & select 'embed' and generate the embed code.  Copy the URL only portion of the embed and change the /embed to /download.  You can then use the modified URL with wget to download the file.

    I've just verified & tested this on macOS, Linux, & Windows.



    spelling correction
    [edited by: PedroPolakoff3 at 11:02 AM (GMT -7) on Jul 13, 2020]
  • What about using a direct download from the software vendors website instead?

    If you have a company website you could host the files from there too.

  • I have this working now with getURL and would like to thank everyone for their responses which helped me.

    After doing a bit more research, it turns out this is a common problem.

    This is the solution fix I have applied, and it is consistent.

    Using OneDrive and after selecting the file in question:-

    1. Select the option to Embed (which will generate some HTML code)

    2. Strip out the unnecessary parts as shown:

    From: <iframe src="https://onedrive.live.com/embed?cid=9EA019747BC00E54&resid=9EA019747BC00E54%21126&authkey=ALr-GsOSko2QFcI" width="165" height="128" frameborder="0" scrolling="no"></iframe>

    To: https://onedrive.live.com/embed?cid=9EA019747BC00E54&resid=9EA019747BC00E54%21126&authkey=ALr-GsOSko2QFcI

    3. Replace the word embed with download in the shortened string.

    https://onedrive.live.com/download?cid=9EA019747BC00E54&resid=9EA019747BC00E54%21126&authkey=ALr-GsOSko2QFcI

     

    I am going to take a look at wGet which I never heard of to see if I can apply this in the future.