I need to schedule my Java app in a Win 2K batch file that will run periodically. Basically, the activity that I need to automate is logging into a webpage with a username and password (submitting a form), then entering some data and submitting another webpage form, and then I'll finally reach the page I want. At that point, I'll scrape the HTML off that page to find the URLs I need to download each member of the listing of large data files.
So, what I don't really know how to do is getting my Java app to pretend it's browsing the internet. I know I need to submit HTTPS POST methods with the correct parameters (for username and password, etc), but I'm not sure how to do that. I know I probably need to use the abstract class HttpsURLConnection in javax.net.ssl, and I see that its superclass has a setRequestMethod, but I don't know how to add my parameters to the POST method. I also don't know how to receive the page that it would respond with in order to parse through it (I can parse--I just don't know how to find that page that it's sending back to me).
By the way, these files and corresponding URLs are virtual and are constructed dynamically, so I can't browse the file system or anything like that--at this point, it looks like the easiest way is to scrape the page for the URLs it gives me to download them. Fun stuff...
Previous Topic
