Re: [unix, console] : something like wget but with forms + authentification

From: Enrique Perez-Terron (enrio_at_online.no)
Date: 09/25/05 

Date: Sun, 25 Sep 2005 16:49:32 +0200

On Sat, 24 Sep 2005 08:55:48 +0200, Thomas Baruchel <baruchel@127.0.0.1>
wrote:

> Le 23-09-2005, Enrique Perez-Terron <enrio@online.no> a écrit :
>> What happens once you are past the login screen? Is your goal just to
>> download the response after submitting your name and password, so you
>> can later pick up that response in any browser, or postprocess it to
>> extract some data fields on the page?
>
> Really sorry for that ; I want to "download the response after

Oh, what have I said? I never meant to be bitchy, there is nothing for
you to be sorry about.

> submitting my name and password", in order to see it in console mode,
> since I do all my work under screen and I don't want to launch an X
> session for the single reason that X web browsers can access to that
> page. I was only wondering if ANY tool of ANY kind (interactive or
> not could manage this single step) ; but the point is that the best know
> tools, like lynx, links, elinks, wget, don't seem to access to this
> "response" (after submitting my name and password) either interactively
> or not. I heard after that of "webfetch", and I think it may be the
> right tool I was looking for, but I didn't manage yet to do it because
> the manual page is not very explicit concerning FORMS though it seems
> webfetch can handle FORMS very well.
>
>> My first thought is "maybe, or maybe not, but don't waste time
>> trying, use Perl or another programming language with similar
>> functionality."
>
> Of course, but I was wondering if a wget-like could do it.

Would you try to save this in a file, and then open it with
links, or lynx, or w3m ? Tell us if it works!

<html>
<head>
   <META http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">
   <TITLE>Identification</TITLE>
</head>
<body>
  <form NAME="formulaire"
action="https://www.cabourse-2.credit-agricole.fr/g1_12/ssl/jsp/identification/1-identifierClient.id"
method="POST">
   <table>
    <TR>
     <TD colspan="3">
      Identification
     </TD>
    </TR>
    <TR >
     <TD colspan="2" nowrap>
      <b>Saisissez votre identifiant :</b>
     </TD>
     <TD nowrap align="right">
      <input type="TEXT" name="identifiant" size="18" maxlength="18" value=>
     </TD>
    </TR>
    <TR>
     <TD colspan="2" nowrap>
      <b>Saisissez votre code confidentiel :</b>
     </TD>
     <TD nowrap align="right">
      <input type="PASSWORD" name="motDePasse" size="18" maxlength="6">
     </TD>
    </TR>
    <TR>
     <TD>
      &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
     </TD>
     <TD colspan="2" nowrap align="left">Pour votre confort, votre
identifiant peut être mémorisé.<br>
      <input type="checkbox" id="cookie2" name="cookie2" value="true" >
      Mémoriser mon identifiant
     </TD>
    </TR>
    <TR>
     <TD colspan="3" align="right">
      <input type="submit" title="Valider votre identification"
name="image">
     </TD>
    </TR>
   </table>
  </form>
</body>
</html>

Save the lines above in a file /xxx/yyy/zzz.html
and say, eg.,

   w3m file:///xxx/yyy/zzz.html

>> Is this the best forum for this question? I am asking honestly,
>> because I don't know offhand a better one. Yet I would think
>
> I asked it to myself also before posting, but after a quick look,
> I noticed that questions concerning wget seemed to be sometimes
> posted here, and since my whole purpose is to avoid graphical
> tools, I finally tought it was relevant ; really sorry if it is not.

I think it is relevant enough; I just wondered if there would be
even more competent people on some other list. In any case, there
are all sorts of questions here and it seems we all do our best
to answer them. You are welcome, no need to be sorry!

>> I guess you would have to use Expect to drive links.
>> How did you go about it with links?
>
> No, I was asking if links could do it ; my version of links can't,
> but maybe a newer version, or maybe with any configuration, but
> using links isn't an issue.

See above. I used wget to download the pages you linked to,
and figured out what those pages did. I am not much of a http/
html expert, but perhaps I have been exposed to a little more
than you, so I gave it a try. It seems to me that the gist of
the second page is that when you click the "Confirmer" button,
the browser shall submit a POST-type request. So I wrote an
ordinary html page with a simplest possible form, where the
input fields have the same names as in the original, and the
submit action points to the same URL as the original.

It appears that the "remember my data" checkbox does not
do anything, but perhaps I misunderstand how cookies are
used. Anyway, it would be cool to hear about it if anything
happens at all. I believe I cannot test it myself since I do
not have a username/password.

>> How much would you be willing to pay a consultant for a complete
>> solution?
> Nothing ;

:)

-Enrique