Designing Web Services at BigBiz

This is a guide to designing web services at BigBiz. It is intended for users that have web accounts at BigBiz.

We will not discuss general HTML or CGI design, nor teach any programming languages like C, Perl, or Java. There are many sources on the web and in various publications that you can refer to, written by people that really know their stuff. For example WebSmith's Links, and The World Wide Web FAQ. We will assume readers of this document are already experienced web designers. If you need more introductory material, please refer to the above link.

Also check out the great index at YAHOO for useful information for beginners through experts. BOOKMARK THIS.

What we will discuss are specifics of designing your web presence with the services at BigBiz, security issues, software available at BigBiz, local "enhancements" you can take advantage of, and other special topics applicable especially to BigBiz web designers. Check this page often, as we will update it as new services become avaiable to our users.

If you are a web designer and you have some tips or tricks to share with our other users, send webmaster@bigbiz.com a note and we will include it (and of course give proper attribution).

We are also collecting questions and answers for a FAQ. If there is something at BigBiz which is confusing, please e-mail webmaster@bigbiz.com, we can answer your question and include it in the FAQ for others.

webmaster@bigbiz.com
The BigBiz Web Design Team

Table of Contents

• Mailing Lists
• BigBiz web server information
• UNIX Manual
• Your Web account
• Your directories
• How to upload/download your files
• Security Considerdations for your files
• Stronghold / Apache HTTP server (web server)
• Secure Web Server
• FTP server
• CGI Scripting
• PHP/FI scripting
• SSI (server side includes)
• Public CGI scripts
• Netscape "file upload"
• Database services
• Access logging and statistics
• Graphics Programming
• Pointers to design aids
  • Icon collections
  • Animated GIF files
  • Java
  • Javascript
  • Backgrounds
  • Image Maps
• Making your web page known to the world
• Restricting your web information from the world
• Password Protection for your web site
• Backups

Mailing Lists

webmasters@bigbiz.com

The mailing list webmasters@bigbiz.com is used to discuss web design issues affecting BigBiz customers. We would announce new software available, changes, design tips, etc... Anyone can subscribe by sending email to webmasters-request@bigbiz.com with the word "subscribe" in the message body. It is highly recommended you subscribe, since announcements of changes which could affect you comes over this channel. If you know other designers that would like to participate, they are also welcome to subscribe.

notify@bigbiz.com

The mailing list notify@bigbiz.com will be used to communicate information about the server: scheduled outages, hardware upgrades, network changes, etc.... You may want to subscribe to this list if you are interested in such matters. To subscribe, send an e-mail message to notify-request@bigbiz.com with word "subscribe" in the message body.

BigBiz web server information

We run Linux UNIX operating system. The web server software is Stronghold Apache SSL. BigBiz has a SSL certificate for secure transactions. We also run wu-ftpd configured for multi-home use, ie. each domain has a virtual ftp server. All users have an anonymous ftp directory you can place public files into.

We have made special changes to the operating system and the server software for extra security - enabling us to offer more services to our users with little or no restrictions as compared to other web-hosting companies. This gives our users significant advantages in placing their web sites at bigbiz.

UNIX Manual

Your Web account

You will also be assigned an Administrative userid. This is used to go into some of our web-based site administration pages. The password is the same as your account password, but you can change it.

You may also receive other user-id's and passwords for accessing other features of our system, like electronic mail and databases.

Your directories

• Mail - used for holding your electronic mail
• html holds your web pages, http://www.yourdomain.com (Personal: http://www.bigbiz.com/~user).
• html holds your secure web pages, https://www.bigbiz.com/yourdomain.
• html holds your secure web pages, if you have your own Verisign Digital-ID https://www.yourdomain.com.
• html/Stats holds your access statistics, generated daily.
• html/cgi-bin holds your CGI scripts, note however that you can place CGI scripts anywhere. http://www.yourdomain.com/cgi-bin.
• pub holds your public ftp files, ftp://ftp.yourdomain.com/pub (Personal: ftp://ftp.bigbiz.com/users/user).
• additional directories may be specified if you use some of our enhanced services.

How to upload/download your files

When you are ready to publish your pages to the web, you can upload the files to your account using ftp. Simply login to bigbiz and upload your files to your web directory (html). A sample session follows, your ftp program may differ.

Sample: we will login to our web account acmewidget (acmewidget.com), and place our web page (index.html) and graphics (logo.gif, SalesTeam.gif) to the html subdirectory. Bold indicates something you type.

% ftp acmewidget.com
Connected to acmewidget.com.
220 web01 FTP server (Version wu-2.4(6) Sun Dec 8 13:19:42 PST 1996) ready.
Name (acmewidget.com:guest): acmewidget
331 Password required for acmewidget.
Password: enter your password here
230-ftp.acmewidget.com: Welcome acmewidget from SHELL.NETCOM.COM !  You are user 3 of unlimited.
230-
230-This is an experimental FTP server.  If have any unusual problems,
230-please report them via e-mail to .
230-
230-If you do have problems, please try using a dash (-) as the first
230-character of your password -- this will turn off the continuation
230-messages that may be confusing your ftp client.
230-
230 User acmewidget logged in.
Remote system type is UNIX.
Using binary mode to transfer files.
ftp> cd html
250 CWD command successful.
ftp> dir
200 PORT command successful.
150 Opening ASCII mode data connection for /bin/ls.
total 2
drwxr-x---   2 acmewidget    acmewidget        1024 Jun  4 22:46 .
drwxr-x---   3 acmewidget    acmewidget        1024 Jun  1 14:57 ..
226 Transfer complete.
ftp> put index.html
local: index.html remote: index.html
200 PORT command successful.
150 Opening BINARY mode data connection for index.html.
226 Transfer complete.
10269 bytes sent in 0.00 seconds (8003.45 Kbytes/s)
ftp> put logo.gif
local: logo.gif remote: logo.gif
200 PORT command successful.
150 Opening BINARY mode data connection for logo.gif.
226 Transfer complete.
1424 bytes sent in 0.00 seconds (2439.69 Kbytes/s)
ftp> put SalesTeam.gif
local: SalesTeam.gif remote: SalesTeam.gif
200 PORT command successful.
150 Opening BINARY mode data connection for SalesTeam.gif.
226 Transfer complete.
32805 bytes sent in 0.14 seconds (222.95 Kbytes/s)
ftp> dir
200 PORT command successful.
150 Opening ASCII mode data connection for /bin/ls.
total 49
drwxr-x---   2 acmewidget    acmewidget        1024 Jun  4 22:47 .
drwxr-x---   3 acmewidget    acmewidget        1024 Jun  1 14:57 ..
-rw-r-----   1 acmewidget    acmewidget       32805 Jun  4 22:47 SalesTeam.gif
-rw-r-----   1 acmewidget    acmewidget       10269 Jun  4 22:46 index.html
-rw-r-----   1 acmewidget    acmewidget        1424 Jun  4 22:46 logo.gif
226 Transfer complete.
ftp> quit
221 Goodbye.
%

You may also publish your web pages directly from your design software (example Netscape Gold), if it has an ftp interface. Here is how you can configure Netscape Gold for BigBiz.

Example: our web account is acmewidget (acmewidget.com). We will place the web design into the html subdirectory (note the /./html in the ftp address).

When you do a "Publish", set the Publising location as follows:

Upload: ftp://ftp.acmewidget.com/./html
User name: acmewidget
Password: enter your password here

We are working on a web-based website maintenance program, more details to come.

We also feature the Microsoft® FrontPage® 2000 extensions on our server, allowing you the option to design your web site using Frontpage 2000 and Frontpage 98. Read more about it. Please check there for support and design tips on using FrontPage with our servers.

Security Considerdations for your files

The unix security model based on user-id is used. The modes on the files should be set for different protections. The group permissions pertain to the http server. We recommend the following modes.

• 0750 - for all directories, this allows you to read and write into the directory, and allows the http server to read the directory. Example: directories under your html directory.
• 0700 - for secure directories, only you (and your cgi scripts) can read and write the directory. Not even the http server can access files there. Example: files containing sensitive information, and your Mail directory.
• 0640 - for all files, this allows you to read/write the file, and the http server to read the file. Example: .html and .phtml files.
• 0600 - for secure files, only you (and your cgi scripts) can read/write the file. Example, your electronic mail.
• 0750 - for CGI scripts, the x bit is set allowing the file to be executed. Example: your .cgi, .pl. and .tcl files.
• 0644 - for publicly accessable files. Example: files in your anonymous ftp site.

When your directories are properly protected, other users will be unable to read or write to your files.

The .htaccess file may be used to further protect selected web directories, example: requiring a valid userid/password before access is permitted. See the .htaccess topic for further information.

Stronghold / Apache HTTP server (web server)

Information about Stronghold and Apache can be found on bigbiz, go to the URL http://www.bigbiz.com/stronghold.html. Additional information can be found on the Stronghold Home Page and on the Apache home page. There is a lot of information there, a good understanding of the server allows you to take full advantage of its features.

In particular we have:

• PHP/FI scripting built in
• CGI scripting enabled
• Built in server side image maps
• Allow Override All - so you can set your own permissions with .htaccess

Secure Web Server

Merchants wanting to do on-line commerce transactions on the internet, like accepting credit cards, should use SSL to make sure nobody can steal the credit card information. Access to the secure web server is included with the domain account, and gives users the tools necessary to build a storefront.

BigBiz has a SSL certificate for https://www.bigbiz.com. Domain based users are welcome to share the bigbiz certificate, your secure web site are accessed with the secure server at https://www.bigbiz.com/yourdomain.

Domain users may prefer to have their own secure site (https://www.yourdomain.com). You can apply for your own certificate and we will install it for you. You may want to check the Verisign Home Page for additional information. We can help you with this process.

To obtain status / configuration on the secure server, visit this link: https://www.bigbiz.com/stronghold.html.

FTP server

Your files are available for ftp access at the host ftp.yourdomain.com, directory /pub. The FTP directory can also be reached using the URL ftp://ftp.yourdomain.com/pub. There is a welcome message in /home/anonymous/yourdomain.com/etc/ftp_welcome that you can edit to suit your needs.

If you are a non-domain user, your public ftp files are accessable on the host ftp.bigbiz.com, directory /users/your_username. The FTP directory can also be reached using the URL ftp://ftp.bigbiz.com/users/your_username.

You can read additional information about the FTP server on the man page wu-ftpd(8).

CGI Scripting

   <FORM METHOD="POST" ACTION="myscript.cgi">

The most important thing you should know is that CGI scripts run with your own user-id. Your scripts can read and write files owned by you, but most importantly: Other users scripts cannot read or write your files! (if you are careful about file permissions).

If you have a file which is accessed by your own scripts, you can set the file mode to 600 so that only you (and your scripts) can read and write the file. Nobody else on the system, nor their scripts, will be able to access your file. This is very important if your files contain sensitive information, like credit card numbers.

BigBiz provides a variety of scripting languages for your use: Perl, Tcl, and Python. In addition you can compose scripts using standard unix scripting languages like sh, csh, awk, etc... You can even run binary programs, like compiled C programs.

The web server configuration will automatically invoke your cgi script if the filename extension is .cgi, .pl, or .tcl, AND your file has execute permissions (mode 750 is recommended for best protection).

Some users prefer to place all CGI scripts in one place. If you like, you can create a subdirectory within ~/html named cgi-bin and place all your scripts there. You can reference your scripts within your html using /cgi-bin, for example (form):

   <FORM METHOD="POST" ACTION="/cgi-bin/myscript.cgi">

Go here to see some example scripts.

PHP/FI scripting

A PHP/FI script can be simply an html file. Scripting commands are embedded within your html. BigBiz webserver will automatically interpret the file as PHP/FI if the extension is .phtml or .php.

BigBiz has made some enhancements to the language we feel would benefit our users.

• We are currently running PHP 3.0.3
• Logging is turned off by default, this is for performance reasons. To enable it you can use SetLogging(1) in your scripts, or use a phpLogging on in your .htaccess
• ShowInfo is turned off by default, most users preferred not to have the stats on the bottom of their pages. You can enable the feature by using SetShowInfo(1) in your scripts, or use a phpShowInfo on in your .htaccess.
• GD support is enabled, max image size is 1024x1024
• PHP scripts run AS YOU, not as the webserver userid. This is a very significant security enhancement. Your scripts can open your files with secure permissions, other users scripts cannot open your files.
• Postgres and MySQL support is enabled.
• InitLog(count [,start_time] ) function is added, used to initialize the Logging file (if Logging is enabled). count is an integer giving the initial hit count. If the optional integer arg start_time is given, then it sets the starting time of the Log (number of seconds since midnight Jan 1 1970).
• MAGIC_QUOTES is not on, this interferes with form processing. Database users will have to do their own quote escaping.
• Basic Authentication passwords are NOT passed to your scripts, this was a security hole which allowed cleverly written scripts to steal passwords.
• Users of older versions of PHP/FI may note that the default setting of XBitHack has changed, and is now OFF. If you desire this feature for your web site, add the line phpXBitHack on to your .htaccess file. Note: the older PHP/FI defaulting XBitHack on was a bug!

SSI (server side includes)

This feature is mainly available for users that have existing scripts. We beleive that PHP/FI offers a much more attractive alternative for new designs.

Public CGI scripts

http://www.bigbiz.com/cgi-bin/Count.cgi

An example would be to embed a link into your web page like this

<IMG SRC=http://www.bigbiz.com/cgi-bin/Count.cgi?df=sample.dat>

Please read the documents before using the counter - there are lots of very cool options available. To keep users from clobbering each other's data files, name your datafile something so it will be unique. Prefixing your domain name is recommended, ie. df=mydomain_mycounter.dat

If you have any questions or want me to install some fonts for you, please email webmaster@bigbiz.com

http://www.bigbiz.com/cgi-bin/forms.cgi

Netscape "file upload"

PHP/FI scripting also has built in file-upload support. Check the PHP/FI docs for additional details.

Database services

Documentation for MySQL can be found at http://www.tcx.se/.

Documentation for Postgres may be found at http://www.postgresql.org. There are several examples of pages on BigBiz that use Postgres

• A database example - working example with source code
• Vivian's Skate Swap Service
• C.O.R.A. Bulletin Board

Access logging and statistics

The file names are yourdomain.com-access_log and yourdomain.com-error_log.

At the end of each month, the log files are archived into subdirectories (YEARMM) and gzipped. Example, the January 1997 log files are in the subdirectory 199701. These archives will be kept for 2 months, after that they may be deleted. If you need the raw log files, please download them to your own system before the 2 month limit.

The information in the log files can provide some important information about who is accessing your pages, which pages, how often, etc..., and can be used for influencing the design of your web site. BigBiz generates up-to-date statistics from the log files for your use.

We run analog 2.0 to analyze the log files. For example, the bigbiz.com reports can be loaded from http://www.bigbiz.com/Stats/. We provide 2 reports:

• thismonth.html - The current month in progress report, including some summaries of the most recent 7 days.
• lastmonth.html - The entire previous month.

Statistics for your domain

You are welcome to run the analog program yourself to get specific information you may be interested in, please read the readme file to see the many options it has.

Or if you want to do more heavy duty analysis of the log files, you should copy them to your own computer and run whatever analysis programs you desire. Also note there is currently no plans to summarize the error log files, you should take a look at these occasionally to see if there are any problems with your web site.

Note: analog only summarizes http requests, it doesnt summarize any file transfers using ftp or e-mail. Secure requests using the shared certificate (https://www.bigbiz.com) are included in the bigbiz reports. If you have your own site certificate, your secure logfiles are in /usr/local/apache/logs/ssl.

Graphics Programming

Pointers to design aids

Icon Collections

We have collected a couple of icon sets which are loaded on BigBiz for your use. Check HERE for details. A great resource for icons is Yahoo.

Animated GIF files

http://members.aol.com/royalef/toolbox.htm contains a comprehensive description on the tools available to design animated GIF files, plus a nice collection too. http://www.aau.pair.com/animated/ contains a pretty large collection of animated GIFs.

Java

What better place to look for Java information than JavaSoft Home Page. You can place your compiled Java applets for download by any Java-compatible browser. No special configuration is needed at BigBiz.

Javascript

Netscape is the home of JavaScript. All you need to know is there.

Backgrounds

Yahoo has a Huge Index of sites that help you design backgrounds that make your pages stand out.

Image Maps

Check HERE for Image Map editors.

Site Checking

Web Site Garage, with an easy-to-use cartoon interface Web Site Garage provides Web site owners with online diagnostic tools to better service their Web sites through promotion and maintenance. By "parking" their URL in the Web Site Garage a seven point diagnostic check will be run on your home page including an analysis of load time, browser compatibility, dead links, link popularity, spelling, HTML design, and several other design areas. The resulting report, presented in just seconds by the wisecracking Web Site Garage mechanic, analyzes and rates the site, providing suggestions for improvement.

Web Site Garage also offers additional services like graphics optimization, and search engine registration.

Making your web page known to the world

Once you've completed your web site, how do you tell the world that it is available? One effective way is to get your site listed on the major search engines on the internet. There are many services that do just that for you, for a small fee they will list your web site. However, it is possible to "do it yourself" for free, and get listed in most of the major search indexes.

These 2 web sites will step you through the process and put you on places like Lycos, Webcrawler, Altavista, Yahoo, and more. Check around, you may find other ways to announce/advertise/promote your web site. And please, make sure you place your web address on your business cards, brochures, advertisements, giveaway T-shirts, etc.... Remember: more exposure is a good thing.

• @ submit
• Submit-it
• Add-me

TrafficBoost.com has a URL submitter to 500 search engines (thats what they advertise). They do charge $49 for this service. We registered with them as a reseller, so if you place your order using this URL: http://trafficboost.com/?BIGBIZ.COM then we get a $9.80 commission. We really dont care much about this income so if you place your order we will credit the $9.80 back to you.* *assuming they are true to their word and mail us a check.

Submit-It has a similar deal, they charge $60 to submit 2 URLs to 400 search engines. We also get a commision that we will credit to your account, so it will only cost you $51. Click here: Submit It!.

Note: we have not used any of those services ourselves and are providing these links only for your convienience.

---

Restricting your web information from the world

There are web crawling robots out there whose job is to automatically surf the web and index what it finds. Many of the popular search engines, like Alta Vista and Lycos, use this technology to create searchable databases of everything it can find on the web. Normally this is a good thing, it makes your site and its contents easier to find. But sometimes you may want to restrict such wide disemmenation of your information. Some reasons may be

• Your content is not ready for worldwide release
• Accessing your files may cause some unwanted side effects, like unwanted incrementing of hit-counters
• Portions of your site contain material which you prefer not to be indexed
• Portions of your site contain duplicated information, or very deep virtual trees.
• Your web site contains information of a topical nature, whose value is lost over time (like headlines from todays paper).

To prevent these robots from accessing some or all of your site, you can create a at the top of your web site (ie. in the html/ directory), called robots.txt, with special instructions for robots. Well-behaved robots first look in this file to see if it should avoid some or all of your site. This degenerate example:

User-agent: *
Disallow: /

will keep all robots off your entire site. The robots.txt syntax allows you to only restrict a portion of your site, or to name which robots to restrict. Read al about it at The Robots Page.

---

Password Protection for your web site

Our server fully supports password protection for part or all of your web site. The machanism uses password files and a .htaccess file in the directory you want to protect. When a user enters a password protected URL, then their browser will popup a dialog box requesting a valid user name and password. Until a valid password is entered, their browser will be denied access to the contents of the directory (and any subdirectories below it)

If you are using FrontPage to design your web site, there are built in features to create password protected sub-webs... we will not discuss this. We discuss how a non-Frontpage user can do it (and in fact, FrontPage simply automates the procedure below).

You will need to create a password database, it can be simply a text file listing usernames and encoded passwords, or it can be a DBM database. We recommend using the DBM database since lookups are much quicker, especially if there are many users. In addition you need to create (or edit) a file named .htaccess in the directory you want to protect and add some lines to it which point to the password file.

First, we create the password file, you need to telnet to the server and go to your directory. (If you do not have a telnet program, you can download one from http://www.hotfiles.com, search for telnet.

Suppose in your html dir there is a subdir named 'private', which you only allow passworded users to go there (and any directories below there). Lets pretend there are 2 users, jim and joe.

We create a password file, lets call it "users":

        cd ~
        dbmpasswd
           - will ask for database name, enter users
           - command, enter A to add a new user
           - enter the users name, jim
           - enter the password, example: rock345
          now jim is in the password file, repeat from the A command to
          enter joe

           - type Q to quit

In your home directory there should be a database file now called users.dir and users.pag. (The two files users.dir and users.pag together are the password database, we refer to this database by the common name users.) You can use the dbmpasswd command to add/delete users in the future.

Now create or edit the .htaccess file (note the filename begins with a dot) in the private directory. Make sure you place this file at the directory you want protected (in this example ~/html/private). You can create this file on your own system then upload it if you like. If there is already a .htaccess file there, edit it so you dont destroy anything else in that file (normally there isnt one but FrontPage users may have one which is created automatically br FrontPage). Add (or edit) these entries

        AuthType Basic
        AuthName MyPrivateArea
        AuthDBMUserFile /home/yourlogin/users
        require valid-user

explanation of each line:

• AuthType - always Basic (until they change browsers to accept other types of authentication)
• AuthName - the name of the realm (or the subweb) under private subdirectory. This is displayed in the dialog box by the browser when a password is requested.
• AuthDBMUserFile - the full path to the password database. Make sure you use the full pathname, starting with /home/yourlogin, replace yourlogin with your login userid. Do not use the .dir or .pag extensions.
• require - there are several options for this, but valid-user means any user in the users dbm file, with a correct password, can gain entry into the subweb.

Now you can test your password site by going to the URL http://www.yoursite.com/private/ , you should get a password request, enter 'jim' and 'rock345'.

Check the apache docs for additional details and options on this feature. http://www.bigbiz.com/stronghold.html, goto html docs, apache, runtime directives.

A useful script is in /usr/src/local/txt2pw, the script takes a file of username,password lines (see the example txt2pw.txt) and creates a password database from it. If you have such a list of passwords that you want to use then you can run it through the script to create or update a password file. Feel free to copy the script and modify it for your own requirements.

---

Backups

BigBiz backs up data on a daily basis. We maintain approx 2-3 weeks worth of backups. However we cannot be held responsible for losses or damages resulting from the deletion or loss of online data for any reason. For your own protection, you should keep your own offline backup.

BigBiz does keep redundant hardware so we can do hardware swaps in case of failure.