I have an applet which launches the jsp in a new window using showDocument method.
When I exit from an applet, all jsps launched from an applet should also get closed.

You would need to code something in JavaScript to close the window but since the jsp window was not opened via JavaScript it's going to be tough to close it. (This is a JavaScript security feature) You can jump through some hoops and get it to work in IE but FireFox, from what I've seen, flat out won't let you do it.

In spite of that limitation here's my suggestion:

Right before you launch the JSP page make a call to a servlet and put a variable in the session called something like applet_open, set the value to yes, true, or something else to indicate that the applet is open.
On the JSP page use DWR (DWR - Easy Ajax for JAVA | Direct Web Remoting) or AJAX to make a call to the server to check that value. If it's not what it should be or the session is null then try to close the window.
Right before the applet closes make a call to the servlet again and change the value.

When you try to close an IE browser with close() expect a message about the window closing, FF3 will not prompt you, it will just return an error from the command. You might want to try something like this:

Code:

---

try {
  window.close();
}catch(e) {
  alert("Please close this window manually.");
}

---