Forum Discussion

YeuHarng's avatar
YeuHarng
Brass Contributor
Sep 04, 2023
Solved

Download Specific Files From Link

https://www.fbi.gov/wanted/topten/alejandro-castillo/download.pdf 

Hi here is the link i need to download the file , between the /topten and /download.pdf is the person name. And the website will update everyday so if i want download all the file how can i write. Cuz if i just use the link https://www.fbi.gov/wanted/topten is will jump out the message:Content-Disposition header not found for URL: https://www.fbi.gov/wanted/topten

  • YeuHarng's avatar
    YeuHarng
    Sep 04, 2023

    Harm_Veenstra 

    Hi, thankieww for u helping but still have error, i'm a newbie in PowerShell, can u explain what does this mean?

     

  • YeuHarng 

     

    I wrote this to parse the parse, quick but this works:

     

     

    $toptenpage = Invoke-WebRequest -Uri https://www.fbi.gov/wanted/topten -UseBasicParsing 
    $urls = ($toptenpage.Links | Where-Object { $_.href -Match 'topten' -and $_.href -notmatch 'History' -and $_.href -notmatch 'faq' }).href |  Sort-Object | Select-Object -Unique -Skip 2
    foreach ($url in $urls) {
        Invoke-WebRequest -Uri "$($url)/download.pdf" -UseBasicParsing -OutFile "c:\temp\$($url.Split('/')[5]).pdf"
    }

     

     

    This will save, create c:\temp if not already present, the top ten people in a .pdf file per person:

     

     

    Please let me know if it works also for you and mark my answer as Solution to mark this topic as solved if it did

    • YeuHarng's avatar
      YeuHarng
      Brass Contributor

      Harm_Veenstra hi sry again to bother u.last time u send me the code, on that day I can run,but tdy I don’t know why can’t can u try on ur site ? 

      • YeuHarng 

         

        Invoke-WebRequest : Just a moment...Enable JavaScript and cookies to continue.

         

        Something changed on that page, and it did work without that... There are other solutions to scrape websites using PowerShell (I just Googled that). Perhaps you can try them?

    • YeuHarng's avatar
      YeuHarng
      Brass Contributor

      Harm_Veenstra 

      Hi, thankieww for u helping but still have error, i'm a newbie in PowerShell, can u explain what does this mean?

       

      • Ah, there seems to be a policy in place on your system. Does it work if you run Set-ExecututionPolicy Bypass and then run the script in the same PowerShell prompt?

Resources