Shalom from Jerusalem, Israel and glad to hear you are considering a visit to Israel!
The best place to cross between Egypt & Israel is at Taba-Eilat and then to Jordan, in the same area at the Arava Crossing (just north of Eilat). In both cases, you don't need a visa to enter any of the countries (you get it on the spot), though, on entering Israel at either Taba or through the Arava crossing, you will get only a 30-day visa (not the usual 3 months), I think, unless that's changed recently.
You mentioned something about "near to Tayiba north of the West Bank" but of course, there's no crossing to Jordan near there, unless you mean the Sheikh Hussein Bridge which is at the northern end of the Jordan Valley, not far from Beit She'an. That's also a good place to cross to Jordan. The most difficult crossing to Jordan is at the Allenby/King Hussein Bridge - there, you have to go through Israeli, Jordanian and PA questioning AND if you are going from Israel to Jordan at that point, you have to have a visa already purchased and in your passport!
And please do NOT, under any circumstances, try to cross at Rafiah (the northern Sinai at Gaza) - not at this time anyway...
You can certainly ask the customs/passport control people NOT to stamp the Israeli visa in your passport - they'll stamp it on a separate sheet, but keep that with you while in Israel, returning it on your departure. Egypt & Jordan will let you in even with the Israeli visa stamp, but other "neighbors" won't be happy about it and may forbid your entry.
Just so you know, Israel doesn't care what visas show in your passport, as long as you are not a terrorist or intend doing any harm while here (in fact, we let anyone in, unlike most of our neighbors...), and yes, you may have to answer a lot of questions and open your bags and even endure a more "invasive" search, depending on where you've been or how you answer the initial questions. But then, we are the target for terrorists and drug smugglers here and are trying our best to protect ourselves from both, as I'm sure you understand.
IN-ISRAEL TRANSPORTION: the inter-city bus system (Egged) is GREAT with buses going everywhere at regular times for a reasonable price. You can also take "sheruts" (shared taxis) between the major cities. And the train system up and down the coast is very good - though the train from Tel Aviv to Jerusalem is probably the slowest of the line... still, it's a beautiful ride along Nahal Soreq and up through the Judean Hills, even if it now arrives out on the edge of Jerusalem near the Jerusalem Mall (which looks like any other mall, except, of course, the store signs are in Hebrew).
Of course, there are buses within each city, and I use the buses in Jerusalem every day, but that's up to you. There are plenty of taxis and, as always, your feet!
If you are interested in guided bus tours around the country, United Tours http://www.inisrael.com/united/ and Egged Tours http://www.egged.co.il/Eng/main.asp?lngCategoryID=2727 offer daily departures from Jerusalem and Tel Aviv to major sites, including the Dead Sea/Masada, the Galilee and more.
Don't forget to spend lots of time in Jerusalem - the center of the universe! As far as walking around at night - well, I do in Jerusalem and feel perfectly safe... there are areas I don't go, but downtown in the Ben Yehuda Street area, in the Jewish Quarter of the Old City and near the Western Wall are all great places to be at night with lots going on and stuff to do, see and enjoy!
If you are coming soon - the annual International Arts & Crafts Fair takes place this month (17-28 August) in Sultan's Pool in Jerusalem. And there's lots more to do in August in Jerusalem, all over town! For more information, go to: http://www.jerusalem.muni.il.
Meanwhile - have a GREAT visit - Israel is a beautiful country and as safe as anywhere in the world these days (probably safest, given our penchant for good security).
And be sure to wr