Issue 5

Access to all regular articles. This page is intended for printing purposes. Note that the internal links to references will not work correctly.

Etc Articles

Software in Use: Externally-Hosted Web Statistics Services

In the second of this regular column which reviews software which is used by Exploit Interactive we report on two externally-hosted web statistics services which are used to provide details of use of the web site.

Background

Exploit Interactive aims not only to provide a dissemination channel for the EU's Telematics For Libraries programme, but also to provide advice and guidance to members of the European library and information community. In particular we can provide advice based on the experiences we have gained in providing this web magazine.

In early January 2000 we introduced two externally-hosted Web statistical services on the Exploit Interactive web site. In this article we report on our experiences.

About Externally-Hosted Web Statistical Services

Most people are interested in web statistics. The statistics may be needed to justify the costs of the service, to monitor where the visitors are coming from, to investigate how the web site is being used or to detect broken links or other problems with the web site.

But it can be difficult to provide access to web statistics to a range of information providers. There may be a cost for the software - although software such as Analog is freely available, software which provides richer functionality may have to be purchased. Once the software has been obtained effort must be spent in installing the software, configuring the web server and managing the log files. Although this can, and should, be automated, it may be difficult to allow, say, departmental information providers to carry out data mining of their data. These difficulties can be compounded for departmental or project web sites, which may not have dedicated IT support.

A number of externally-hosted services for providing web statistics are available, including NedStat [1], SiteMeter [2], SuperStats [3] and Stats4All [4]. Such services can eliminate any licence costs, and can minimise ongoing maintainace costs.

These service typically work by providing a small icon which is included on pages on the web site. When a visitor accesses a page containing the icon, a request for the image is sent to the company hosting the service. The request is logged. The log file may store not only the date and time of the request, but also a variety of additional information, such as the referer (sic) field (how did the visitor get to the page) and the search engine query string (if the visitor used a search engine to get to the page, what search term did they enter).

In addition to information which will be recorded on the local web server's log file, the externally-hosted services often include Javascript code which can interrogate the visitor's client machine and obtain information on the operating system, browser, screen resolution, etc).

Arguably one of the advantages of externally-hosted web statistical analysis services is the ease of access to the reports. Typical clicking an icon (such as the icon to be found near the bottom of the page) will provide access to the information. The Exploit Interactive statistics provided by SiteMeter are freely available [5] and examples of the reports are shown below.

SiteMeter

SiteMeter is used to provide site (as opposed to page) statistics. Exploit Interactive has been using SiteMeter since 4 January 2000. Figure 1a gives a graphical representation of the numbers of page views and visits during February 2000. Figure 1b shows the browsers and browser versions which have accessed the web site.

Figure 1a: Numbers of page views and visits in February 2000

Figure 1b: Pie chart of the browsers and browser versions which have accessed the web site

Other features of SiteMeter include:

• Total and average numbers of visits and page views over the last hour, current day and week, together with graphical displays showing the trends.
• Information (IP address of client, number of pages viewed, length of visit) of current visitors to the web site.
• Details of the "referer" (sic) field for recent visitors (i.e. the link followed by visitors to the web site). This can be useful for finding the search terms used. Since the link back to the search engine can be followed, this can also be useful in determining the position of your web site in the search results.
• User profiles, including information on the time zone, operating system, browser, domain of the users client machine.

In addition to the graphical display a weekly email summary can be received, as shown below.

                    Exploit Interactive
                    (sm2-exploit-home)

                    -- Site Summary ---
          Visits

            Total ........................ 5,234
            Average per Day ................. 71
            Average Visit Length .......... 3:02
            This Week ...................... 436

          Page Views

            Total ....................... 14,010
            Average per Day ................ 189
            Average per Visit .............. 2.7
            This Week .................... 1,010

  http://sm2.sitemeter.com/stats.asp?site=sm2-exploit-home

                  --- Visits this Week ---
       Day
Hour  3/10   3/11   3/12   3/13   3/14   3/15   3/16   Total
---- ----- ------ ------ ------ ------ ------ ------ -------
...
 9       6      0      1      1      8      3      3      22
10       2      1      3      4     11      5      4      30
11       7      2      8      3      5      7      7      39
12       5      1     12      5      6      6      4      39
13       3      1      4      4      5      4      1      22
14       3      3      1      4      0      0      7      18
15       0      2      1      2      1      2      8      16
16       0      1      1      4      2      0      4      12
17       0      0      1      4      2      7      4      18
...
    ------ ------ ------ ------ ------ ------ ------ -------
        56     21     40     79     91     69     80     436

                --- Page Views this Week ---
       Day
Hour  3/10   3/11   3/12   3/13   3/14   3/15   3/16   Total
---- ----- ------ ------ ------ ------ ------ ------ -------
...
 9      11      0      1      3     11     17     12      55
10       4      2      4      5     17     10      5      47
11      12      7     11     10      8      8     11      67
12      14      8     23     10      8     12      6      81
13       8      1     10      7     16      6      1      49
14       7      3      1      7      0      0     12      30
15       0     13      2      3      1      2      8      29
16       0      1      2      7      4      0      5      19
17       0      0      1      6      2     12      7      28
..
    ------ ------ ------ ------ ------ ------ ------ -------
       137     47     69    162    157    213    225   1,010

SiteMeter provides a management interface which can be used to customise the display. It also allows the public display of the information to be suppressed.

NedStat

Exploit Interactive has also been evaluating the NedStat externally-hosted statistical service. With NedStat a unique id is used to enable statistics for individual pages to be provided. An example of output from NedStat is shown below. The summaries are taken from the statistics for the article on "Building Europe's Largest Library", published in Exploit Interactive issue 4 [6]. Access to the latest statistics is also available [7].

Statistics from 7 January 2000, 10:00
Compiled on 17 March 2000, 09:16

Total number of pageviews: 795
Top day so far: 14 January 2000 with 179 pageviews

Figure 3: Nedstat Pro Output (portion)

Another example of use of NedStat can be see at the Information Research e-journal web site [8]. This web site has been using the service to record visits to their home page since 1 April 1998. From that date up to 17 March 2000 NedStat had recorded 37,783 page views - an indication that the service can provide information on long-term trends.

Nedstat Pro

The versions of SiteMeter and NedStat described above are both available for free. Their business model is based on an income stream from the advertisements which are displayed when you access the service and licence fees for more fully-functioned versions of the software.

NedStat kindly donated a copy of their NedStats Pro service [9] for evaluation purposes. A brief summary if given below.

With NedStat Pro an invisible icon is placed on up to 25 key pages. Since there is no obvious icon to click on the data is not available to visitors. Access to the data is available by a URL, and, as an additional security feature, a username and password must be given.

An example of the information provided by NedStat Pro can be seen in Figure 4.

Figure 4: NedStat Pro Interface

Useful features of NedStat Pro include the ability to specify date ranges and being able to output results to comma-separated variable (CSV) files for post-processing locally.

Discussion

As mentioned in a recent Ariadne article [10] there is growing interest in the use of externally-hosted Web services. But are remote services reliable? Ellen S. Gordon, Nedstat's Marketing Communications Manager, provided answers to the following questions.

What happens if your services goes down?

Of course, this is a concern for customers - we do our absolute best to get our service up as quickly as possible and minimize down time.

What happens if the network to your service goes down?

This is of course external to our systems, but again a high priority to us. We are at the moment changing to a different ISP because we feel we've found one with even more reliability than our current one. Also, there is some redundancy built in to the network.

Do you have any comments regarding access by browsers which do not download images (e.g. text-only browsers, browsers with images switched off)?

These are a very tiny percentage of users, but are not counted.

Is it possible to upload the statistics to my computer, for further analysis locally?

It is possible to download CSV files into a spreadsheet to further analyze and manipulate the data.

What are the performance penalties associated with your service?

Extremely minimal - it's possible that at times page loading might take a little longer because of image loading.

Exploit Interactive is based in the UK. Do you have any plans to provide a Nedstat service in the UK, to avoid network traffic to the US?

Your service is actually coming out of the Netherlands, and it does not negatively impact your service in any way to have it located outside UK. As for representation of the company, we are currently opening an office in London.

There are an increasing number of third party web statistics services. What makes NedStat special?

Brian, the key thing to start off with is that, while many such services rely on log file analysis to deliver their information, we have developed a technology that allows for virtually instantaneous data delivery. This real time information is a tremendous asset to website managers. Also, as you use and explore Pro I'm sure you will find that the depth and breadth of information it captures is impressive. The number of categories of information, and the customization that is possible with them,gives a tremendous amount of data with which to work when analyzing site effectiveness. Some of these are: detailed listings of referring links, search engines, and search words; visitors' city and country of origin; specs on visitors' OS, browser, and ISP . . . and much more. I hope this answers your questions, and that you will enjoy using NedStat Pro! Let me know if there is anything else we can help you with!

Conclusions

How useful are externally-hosted Web statistics services such as Nedstat and SiteMeter? For a web magazine such as Exploit Interactive they appear to be very useful. From a management point of view we require site statistics. However individual author's of article will be more interested in accesses to their article. A combination of SiteMeter's and Nedstat's services enable both of these requirements to be satisfied.

These service, do, of course, have their limitations. They will typically only provide detailed information for a short period, for example, the last month, and it is not normally possible (with the free versions of the services) to carry out "data-mining". They will also not provide information about, say, errors which will be available in server log files. The services also have a dependency of the availability of the remote service and the network. This may be a concern to some, although as network performance and reliability increases the dangers should be reduced.

An important point to be born in mind is that meaningful web statistics can be difficult to obtain, as described elsewhere in this issue [11]. Externally-hosted Web statistics services are dependent on the use of graphical icons. Text browsers, browsers with images switched off and robot software will not be recorded by these services. However, since the images are non-cachable objects, these service should overcome the problem with missing hits which affects server log files.

How do the figures provided by these services compare with figures produced by conventional web statistical analysis packages? This is a question we intend to address in the next issue of Exploit Interactive.

References

1. Nedstat
   URL: <http://uk.nedstat.net/>
2. SiteMeter
   URL: <http://www.sitemeter.com/>
3. Superstats
   URL: <http://v2.superstats.com/>
4. Stats4All
   URL: <:http://www.stats4all.com/>
5. Exploit Interactive Website Statistics, SiteMeter
   URL: <http://sm2.sitemeter.com/stats.asp?site=sm2-exploit-home>
6. Building Europe's Largest Library, Exploit Interactive, issue 4, January 2000
   URL: <http://www.exploit-lib.org/issue4/ell/>
7. Statistics For "Building Europe's Largest Library", Nedstat
   URL: <http://uk.nedstat.net/viewstat.asp?name=exploit-4-ell>
8. Information Research, University of Sheffield
   URL: <http://www.shef.ac.uk/~is/publications/infres/ircont.html>
9. NedStat Pro,
   URL: <http://uk.nedstat.net/pro.html>
10. Using Externally-Hosted Web Services, Ariadne, issue 23
    URL: <http://www.ariadne.ac.uk/issue23/web-focus/>
11. Performance Indicators For Web Sites, Exploit Interactive, issue 5, April 2000
    URL: <http://www.exploit-lib.org/issue5/indicators/>

Author Details

Brian Kelly
UK Web Focus
UKOLN
University of Bath
Bath
England
BA2 7AY

URL: <http://www.ukoln.ac.uk>
Email: b.kelly@ukoln.ac.uk

For citation purposes:
Brian Kelly, "Software in Use: Externally-Hosted Statistical Software", Exploit Interactive, issue 5, April 2000
URL: <http://www.exploit-lib.org/issue5/software-used/>

Audit of Links on the Exploit Interactive Web Site

In a follow-up article we give an updated audit on links on the Exploit Interactive web site.

Background

On 22 December 1999 an analysis of the Exploit Interactive web site was carried out using Microsoft's SiteServer Analyzer package. A summary of the findings was published in Exploit Interactive issue 4 [1]. How have things changed since then?

Update Of Audit

On 28 March 2000 the Microsoft's SiteServer Analyzer package was again used to analyse the Exploit Interactive web site. We were pleased to find that no significant broken links were found (a handful of links which were deliberately broken in order to demonstrate 404 error message were found).

Although no broken links were found, the analysis did report on a number of unusual error codes. On further investigation these were found to be the result of server redirects. The conclusion for the SiteServer analysis seems to be that the Exploit Interactive web site contains no significant broken links.

Xenu Analysis

In order to confirm the positive report from the SiteServer analysis a second link checking tool was used. We made use of the Xenu package [2].

Xenu reported on several links which were not available. These were not available for a variety of reasons including (a) access was forbidden, (b) resource was not found, (c) the service was temporarily overloaded or (d) the remote service was unavailable.

Xenu detected many more links than SiteServer. This was partly due to the limited configuration options provided in Xenu which means that it is not possible to suppress link-checking in named areas. This is needed for the Exploit Interactive web site, as it contains a number of reports of analyses of remote web sites which contain information on broken links.

Xenu also detected invalid internal links in the "Print Entire Issue" section. This page provides the entire content of an issue in a single page. It is intended for users who wish to print the entire contents of an issue. The page is created by a server-side include file which pulls in all article fragments, excluding navigational elements. A side-effect of this is that internal links (e.g. to references) will not work, as there will be duplicate link names.

Xenu found about a dozen legitimate broken links which were not detected by SiteServer. A number were due to invalid HTML which, unfortunately, had not been spotten when the issue was released. These errors were fixed. There were also about half a dozen links which were no longer available. These were typically links to a conference or news web site, and the resource appears to have been deleted.

Policy On Correcting Broken Links

Once we have found a broken link in a published article, we have to decide what to do. If we make a change we could be accused of retrospectively altering a published document. This may be a dangerous precedent to set. On the other hand the broken link will cause problems to users who try to follow it, and will make future link checking more difficult as the numbers of broken links grow.

We have chosen a compromise. Since the hose style for hypertext links is to provide them in the references for an article, and to display the URL as the hypertext anchor, we can remove the hypertext link, leaving the URL as plain text. This provides the full meaning provided in the original article, but avoids users wasting their time in following links which are known to be broken. As a reinforcement an icon is provided which indicates that a link is broken, as illustrated in Figure 1.

Figure 1: How Broken Links Are Treated

Discussion

In addition to the two packages mentioned above, the server log files were also examined briefly. It was noticed that there were a number of 404 error messages present. Further examination revealed that most of the errors were due to missing resources which were called from style sheet files. This is believed to be the result of malformed URLs. For example if somebody creates a link to this article in the form <a href="http://www.exploit-lib.org//issue5/exploit-audit/"> (i.e. with a spurious slash [/] in the URL) the page will appear to be displayed correctly, but links to images in style sheet files will fail.

Report Of Findings

Although the SiteServer analysis appears to miss some broken links, it does provide a useful summary of the extent of a web site. The summary of the findings is given in Table 1.

Note: The number of links excludes links contained in the report of the web site.

It will be noted that the number of pages appears to have gone down. This is due to the removal of a number of incorrect links to resources which meant that in some cases resources were duplicated (e.g. http://www.exploit-lib.org/issue1/amazon/ and http://www.exploit-lib.org/issue1//amazon/ were treated as two separate resources).

Conclusions

The comparison of the two link checking packages appears to show that the Xenu link checker is more comprehensive, although SiteServer provides better control over excluding resources. However both packages fail to provide information on broken links with style sheets.

References

1. Audit of Links on the Exploit Interactive Web Site, Brian Kelly, Exploit Interactive issue 4, January 2000
   URL: <http://www.exploit-lib.org/issue4/exploit-audit/>
2. Xenu,
   URL: <http://www.xenu.com/>

For citation purposes:
Brian Kelly, "Links To Telematics For Library Web Sites", Exploit Interactive, issue 5, April 2000
URL: <http://www.exploit-lib.org/issue5/exploit-audit/>

Links To Telematics For Library Projects

In this review of Telematics For Library Project web sites Brian Kelly reports on the numbers of links to the project web sites.

Background

Exploit Interactive issue 4 included an article which describes ways to promote your project web site, and in particular, ways of getting your web site indexed [1].

If you follow the advice given in the article, how will you know if you have been successful? One solution is to systematically record the numbers of links to your web site. If, over a period of time, you notice that the numbers of links to your web site is growing, you can be confident that people are noticing your web site, and value it sufficiently to make links to it. If, on the other hand, the numer of links to your web site fails to grow, this may indicate problems with your web site or with your dissemination strategy. If you know you have problems, you will be in a position to do something about it.

Finding Out About Links To You

The web as originally designed provided one-way links. There was no way of finding out about links to your web site. However there are a number of ways in which you can obtain such information.

• Analysis of server logs: You can analyse your server logs. The "referer" field will provide information on the page a user was viewing when they followed a link to your web site.
• Analysis of search engine databases: Several of the large search engines allow you to find links to a web site.

In this article we use the second option.

Report

A survey of the number of links to Telematics For Libraries projects (as listed at [2]) was carried out on Friday, 7th April 2000.

Discussion

An article elsewhere in this issue of Exploit Interactive [3] reviews a number of performance indicators for web sites, and includes links to a web site as a possible indicator. The author has carried out an analysis of links to web sites within the UK Higher Education community and has reported the findings in the Ariadne web magazine [4].

As mentioned in the Ariadne article, the numbers of links provided by Alta Vista appear to fluctuate. This phenomena has been reported by others [5]. There may be several factors which affect document fluctuation; for example the fluctuations in AltaVista are apparently due to variations on the the load on AltaVista's server farm.

Questions as to the relevance and reliability of web-based link data have still no been resolved. The author welcomes comments on these questions.

References

1. Promoting Your Project Web Site, Brian Kelly, Exploit Interactive issue 4, January 2000
   URL: <http://www.exploit-lib.org/issue4/promotion/>
2. Telematics for Libraries - Projects
   URL: <http://www.cordis.lu/libraries/en/projects.html>
3. Performance Indicators For Web Sites, Brian Kelly, Exploit Interactive issue 5, April 2000
   URL: <http://www.exploit-lib.org/issue5/indicators/>
4. A Survey Of Links to UK University Web Sites, Brian Kelly, Ariadne issue 23
   URL: <http://www.ariadne.ac.uk/issue23/web-watch/>
5. Internet Search Engines: Fluctuations in Document Accessibility, Mettrop, W., Nieuwenhuysen, P., and Smulders, H. In Proceedings of Internet Librarian International 2000, 20-22 March 2000, London, UK. Edited by Carol Nixon and Jennifer Burmood, Information Today Inc, pp. 153-163

Author Details

Brian Kelly
UK Web Focus
UKOLN
University of Bath
Bath
England
BA2 7AY

URL: <http://www.ukoln.ac.uk>
Email: b.kelly@ukoln.ac.uk

Brian Kelly is UK Web Focus. He works for UKOLN, which is based at the University of Bath.

For citation purposes:
Brian Kelly, "Links To Telematics For Library Web Sites", Exploit Interactive, issue 5, April 2000
URL: <http://www.exploit-lib.org/issue5/telematics-links/>

Issue Home | Editorial | Features | Regular Columns | News and Events | Et cetera
Go to Top	A UKOLN Service. Contact Us. Copyright © 1999	Last Updated: 7 April 2000