cooking, programming and everyday life xrado

Monday, October 27, 2008

FireSpider extension for Firebug

FireSpider is a an extension for Firebug that can help you find broken or misleading urls and optimize your web page for search engines.

Extension enables you start and the stop the spider. After pressing the start button it fetch the current url and parses the content for new urls. Every unique url is fetch only once. It follows only the current domain urls and detects its content type. Not html content types are reported. Besides fetching, firebug panel gives you information about page title, page where url was got from, its link name and loading time. Reported are also not found urls and ones went in time out after 10 seconds. Requests are send one after another, so some sites may find it to aggressive or may detect you as a dangerous script. Be careful.


This is the first version (hope not the last) and first extension i ever made. Some things may not be very well done, but the basic functionalities are working, i think. Try it and tell me what you think. Hope you find it useful.

firespider-0.1.xpi ( download / install )

FireSpider on GitHub
Great script ... very useful ...

Awesome extension... tnx dude!

16:43, Monday, November 10, 2008 Paul Davis, from

Very Nice.

I just tried it on a phpBB topic/post with 2 pages and it really took off ! I guess it finds all the links on all linked pages ??? (I'm just learning how to use it). It took quite a while, but it's a huge phpBBS.

Is there anyway to export the results besides SELECT ALL and COPY ?

Anyway to restrict it to only the current page in the browser tab/window ?

Anyway it looks like a nice add-on. I'll have to keep working with it.

yes it finds links from the whole page. no for now is not possible to export the results, may be in future. what kind of report would you need and why? yes i'm planing to add more features, like current page only, internal && external urls, ... i just need some free time.

Hey xrado!

I was just looking at the .js files you included in the .xpi, I'm new to Javascript so I'm looking at your work to see how to do XMLHTTP requests and to wait for the pages to load, etc, etc.

For exporting it'd be nice just either to list the urls it finds on the page as plain text, or with the data you provide as CSV type information. Be nice to copy the export to the clipboard, since Firebug can copy HTML to the clipboard. Just ideas at this point.

I can grab URLs from web pages and websites using various tools, like wGET, CURL, XMLStarlet and automating MSIE via the MSHTML object model; also tools like WinHTTRack can spider and download web sites.

The hard part is downloading all the pages in a phpBBS or a vBulletin topic. Ever been to a bbs and the topic you want is 20+ pages !!! ?

So I've written scripts that will download all the pages in that one topic and then rewrite the navigation links so that the local pages can now use them.

Things like that.

Anyway, I don't want to heap lots of work on you; just ideas at this point. However the ability to examine all the links on the tab or page I'm viewing and export those would be very nice ! Sure beats having to write a script each time !

Very nice add-on and I will hopefully learn a lot from the hard work you've put into it.

Thanks again !

20:58, Wednesday, November 12, 2008 Mr. T, from outerspace
Great tool and congratulation with the .1 release. I got it through the Firebug Google Group.

Exporting the result to a csv or similar format would be great.
I guess your target audience would be expanded( like bosses and similar who likes to put things in a presentation etc.).

+1 for limiting the run to links on a single page.


Nice plugin. It's exactly what I'm looking for testing my application.
But there are some features for improvement:
1) Options for specifiyng ignorable "GET" parameters.
2) Support of pages in UTF8 (where characters encripted as &#XXX;)

If you would like, I can help you with development of this plugin.

hi there! :) I am glad that you like it and you are welcome to help developing it. Since the first release, I've been busy with other stuff, so I didn't do anything more, but I like to. feel free to contact me. greetings.

14:39, Friday, March 06, 2009 Vadim P., from
Quite useful thing, thanks. I hope you'll get time to improve it :)

00:29, Thursday, March 19, 2009 sic_people, from
Hello i was trying the plug in and found a bug, when you try to access to an URL via other port besides the 80 port, firespider try to access all the URL through the 80 port.

thx for info!

nice plugin, but won't work with frames.

Firespider does not respect the Base-Tag which specifies the default prefix for all (internal) links on the page, e.g.

not compatible with FF 8.0.1. kindly update. thanks

Not compatible with FF 10.0.2

20:15, Friday, August 31, 2012 root, from Moscow
Not compatible with FF 15

19:14, Saturday, May 04, 2013 Daniel, from Lagos
Out of date. Please upgrade. Thanks

I think, you are merged with uncle G?

07:36, Monday, June 23, 2014 Azee, from
This addon is not compatible with latest version of firefox. Will you please make it compatible with the latest version of ff.

Not compatible? Why not, too hard?

10:43, Thursday, March 30, 2017 hiпuruh, from заказать продвиж
заказать прогон хрумером логин скайпа pokras77777

23:05, Monday, April 03, 2017 Randybum, from смотреть фильм Та
2070 год. Москва. Мир лежит в руинах после ядерной войны.
Под обломками и пеплом, заполнившими город, всё еще пытаются выживать люди,
несмотря на бедность и постоянный страх.

смотреть здесь

Norvasc Active Ingredient: [b]amlodipine[/b] is used for treating high blood pressure and angina (chest pain).

Amlodipine (Norvasc) is used for treating high blood pressure and angina (chest pain). It may be used alone or with other medicines. Norvasc is a calcium channel blocker. It works by relaxing (dilating) your blood vessels, lowering blood pressure, [url=]Buy Online Almodipine[/url] and decreasing heart rate, which lowers the workload on the heart. It also dilates coronary arteries increasing blood flow to the heart.


TAGS: [b]Norvasc 10 mg[/b], [b]Norvasc 5 mg[/b]

Purchase Almodipine Online
Buy Almodipine Online Without Prescription
Over The Counter Almodipine
Buy Almodipine Online Without Prescription
Online Almodipine
Where Can I Buy Almodipine
Over The Counter Almodipine
Over Counter Almodipine
Online Almodipine
Almodipine Online No Prescription
Where Can I Buy Almodipine
Ordering Almodipine Online

22:31, Friday, April 28, 2017 Jamesfluex, from заказать продвиж
прогон хрумером логин скайпа kai230361

23:02, Tuesday, May 02, 2017 StevenDom, from заказать прогон х
заказать прогон хрумером логин скайпа kai230361
[url=]заказать продвижение интернет ресурса[/url]

11:27, Tuesday, May 16, 2017 StewartGic, from
wh0cd979833 [url=]Buy Cialis[/url]

[b]Stromectol[/b] is used for treating infections caused by certain parasites. Stromectol is an anthelmintic. It works by killing sensitive parasites.

Stromectol as known as: Ivectin, Ivenox, Ivera, Ivergot, Ivermec, Ivermectina, Ivermectine, Ivermectinum, Ivert, Ivexterm, Kilox, Mectizan, Quanox, Revectina, Scabo, Securo, Simpiox


TAGS: [b]buy Stromectol online[/b], [b]Stromectol 3 mg[/b]

online stromectol no prescription
stromectol without prescription
[url=]buy">]buy prednisone usa online cheap[/url]
purchasing stromectol online
buy stromectol no prescription
where to buy stromectol
[url=]buy">]buy prednisone usa online cheap[/url]
stromectol purchase online
stromectol online
[url=]order cheap cytotec online[/url]
order stromectol no prescription
stromectol buy
buy stromectol on line
[url=]order neurontin in USA[/url]
buy stromectol on line
buy stromectol online without prescription
stromectol online uk
where can i buy stromectol
[url=]buy augmentin online no prescription[/url]
order stromectol online
buying stromectol online

What do you think?
name from url
   can't read the code? (reload)