THE WELCH COMPANY
440 Davis Court #1602
San Francisco, CA 94111-2496
415 781 5700


S U M M A R Y


DIARY: December 5, 1997 00:05 AM Friday; Rod Welch

Changed HTML conversion to run continuously; improved line numbering.

1...Summary/Objective
2...Segmented and Consolidated HTML Conversion Code
.....Minimize Using E Command to Avoid Memory Leak
3...06116 - Change to DOSE command so it will run longer
.....Replace Macro 1201 with DOS Record Selection No-Dup Keys
.....Convert records in groups of 20 using DOSE - Process 3K Records
.....Less than 20 Records Processed by 06116 and 070405 directly
4...Numbering Executive Summary, Action Items
5...0704050 - Format SDS Record as Memo and Add HTML
.....Two Formats: "Memo" and "Data Base"
.....Conventional Numbering for Executive Summary
.....Avoid Linking Exec Summary Line to Another Exec Sum Line
.....Sub-level Outline Support in Summary
....."Summary/Objective" on Primary Outline Level in Exec Summary
.....Highlight Action Items and Link Anchors
.....Avoid Link ID Across SDS Line Number Field
.....Line Number Filed, Avoid Linking Split Citations
.....Remove Heading Underlines
.....Link Anchors - Make Distinctive
.....Citations on a Heading Line
6...000007 - Changed Selection Process to "Do-Duplicate" Keys
7...Contacts Suppress Personal Phone and Cellular Phone Numbers


..............
Click here to comment!

CONTACTS 

SUBJECTS
Configure for network s: drive
Intranet, HTML Code
Memos, 022F05, and ASCII for Internet
Memo Convert SDS Record to HTML for
HTML Convert Record and Save to HTML Data
SDS Data Base, Appearance

1108 -    ..
1109 - Summary/Objective
1110 -
111001 - Follow up ref SDS 8 0900, ref SDS 7 5194.
111002 -
111003 - Made 070405 the processor for HTML conversion.  It is called from the
111004 - "Report" menu to convert an SDS record into a memo submission via
111005 - Internet, as an attachment to a letter.  070405 can be called from the
111006 - "File" menu to save an SDS record into the HTML data base.  070405 is
111007 - also called by 06116 to convert a specified group of SDS records for
111008 - the HTML data base.
111009 -
111010 - Changed scheme of Executive Summary and Action Items to use
111011 - conventional line numbering, and eliminated the secondary memo format
111012 - that was initially used.
111013 -
111014 - Changed algorithm in 06116 and 000007 for making data base conversion
111015 - so it runs continuously, processing 20 records at a time, ref SDS 0
111016 - 3255, although there may still be a limit of several thousand records
111017 - at a time.
111018 -
111019 - It is difficult to achieve a uniform representation of SDS records
111020 - with HTML coding.  I made some progress today.  On my Web netscape
111021 - browser, the records seems to give a consistent appearance.
111022 -
111023 -     [See explanation of entire system on 971206, ref SDS 11 6000.]
111024 -
111025 -     [Created "Update" feature on 971208. ref SDS 12 0630]
111026 -
111027 -     [On 971209 made improvements recorded in this record, ref SDS 13
111028 -     4747.]
111029 -
111030 -     [On 980311 improved update feature. ref SDS 16 4880]
111031 -
111032 -     [On 980324 improved appearance of SDS records as Web pages. ref
111033 -     SDS 17 5765]
111034 -
111035 -     [On 980325 had step 20 down to 5, to get this to run really
111036 -     continuously. ref SDS 19 1368]
111037 -
111038 -
111039 -
1111 -
1112 -
1113 - Progress
1114 -
111401 -  ..
111402 - Segmented and Consolidated HTML Conversion Code
111403 -
111404 - Up until today, all of the work has been done primarily in 0704013,
111405 - ref SDS 5 3442, and 070405. ref SDS 5 6666  Initially, 0703 and 07031
111406 - were combined with 0704013 to form a processor that created an
111407 - Executive Summary and Action Item front end to SDS memos.  070405 was
111408 - created to add the HTML coding.  I then combined 0704013 and 070405
111409 - with 06112 to create 06116. ref SDS 4 1327  070407 was created on
111410 - 971206, ref SDS 1 3557, to save individual records to the HTML data
111411 - base. This resulted three separate processors 06116, 070405 and 070407
111412 - that each converted SDS records into a memo format with HTML for
111413 - different situations.
111414 -
111415 - This was impractical to maintain for the reasons below. ref SDS 0 1485
111416 - I found that getting HTML to produce a desired display is largely
111417 - experimental, and so with three different macros, all trying to
111418 - produce similar but slightly different results, it was too complex.
111419 - Observing a problem in one setting resulted in adjustments that then
111420 - had to be replicated in the other two.  Adjustments then had impacts
111421 - that required further changes in the other two and these changes were
111422 - impossible to track.
111423 - ..
111424 - A second major problem was that 06116 was unable to process more
111425 - than month or so of records, 50 to 100, due to memory limitations.
111426 - This meant it would take too long to create the record on any
111427 - particular matter. ref SDS 7 0860  When it takes a long time to do
111428 - something, then there is greater tendancy to accept a lower level of
111429 - performance.
111430 - ..
111431 - Today and over the ensuing weeks I fixed these problems.  For
111432 - convenience, I am reporting all of the work in this record.
111433 -
111434 - I added an option under -entry 120 in 000007 by adding a no-duplicate
111435 - keys condition, per below, ref SDS 0 2565, so we do not have to remove
111436 - the duplicates in memory with macro 1201, as was done up to now
111437 - because the subject selection process in 06112 needed the duplicates
111438 - and it was the initial model for creating this code. Actually on
111439 - 971215, I realized the advantage of making the change in the selection
111440 - step rather than in 06116 with macro 1201.
111441 -
111442 - I made 06116, 070405 and 0704013 separate macros that are modularized
111443 - according to function.
111444 -     ..
111445 -     070405 is the only processor to convert an SDS record into a
111446 -     memo format and add HTML.
111447 -
111448 -     06116 still uses the 06112 front end to select records for
111449 -     conversion to HTML, but now it calls 070405 to format selected SDS
111450 -     records into memo format and add HTML code.
111451 -
111452 -         [Later code was added to perform an "update" function. ref SDS
111453 -         10 1428]
111454 -     ..
111455 -     0704013 has gone back to merely producing an ASCII file that
111456 -     can be used as an alternate form of sending SDS records by email,
111457 -     for people who do not have Web Browsers.
111458 -     ..
111459 -     On this date (today) I noted that it is attractive for
111460 -     maintenance purposes to use 0704013 as a macro to uniformly format
111461 -     SDS records for HTML, and to use 070405 to uniformly add HTML
111462 -     code.
111463 -
111464 -         [On 971209 made major effort to conform 06116, 070405 and
111465 -         070407, per ref SDS 1 1554.]
111466 -
111467 -     The problem is that for each application, we need some separate
111468 -     stuff, and when each of these is called in 06116, the file is so
111469 -     big it cannot sustain processing of a lot of SDS records.
111470 -
111471 -     So, I will probably regret it, but I made these separate macros
111472 -     with a lot of duplcated code for creating the memo format and
111473 -     adding HTML with the aim of producing a uniform looking SDS HTML
111474 -     record.
111475 -
111476 -     ...I did regret this approach, and so over the next several weeks,
111477 -     and on 971213 in particular, I finally got 06116 so it runs as
111478 -     needed by making 070405 the only HTML processor.  06116 now only
111479 -     calls 070405 after selecting the files to be processed.
111480 -
111481 -
111482 -      ..
111483 -     Minimize Using E Command to Avoid Memory Leak
111484 -
111485 -     A basic objective of this effort was to minimize using the e
111486 -     command to open a new file.
111487 -
111488 -     I think this may be another source of memory leak, along with
111489 -     using the cut file, which was changed on Wednesday. ref SDS 7 6360
111490 -
111491 -     The idea is to open a file once and then use the get command to
111492 -     avoid having to create the 2K setup for the file continuously.  So
111493 -     we use "stufq" as a temporary file to read the records in.
111494 -
111495 -     I am only guesing this impacts performance.  It have no impact.
111496 -
111497 -
111498 -
111499 -  ..
111500 - 06116 - Change to DOSE command so it will run longer
111501 -
111502 - Changed much of the design.  Removed all code from 0704013, 070407 and
111503 - 0704012.  The following change to 06116 in combination with the change
111504 - to 00007 adding a no-duplicate keys command, described below, ref SDS
111505 - 0 2565, enable this process to run for a year or so.
111506 -
111507 -
111508 -      ..
111509 -     Replace Macro 1201 with DOS Record Selection No-Dup Keys
111510 -
111511 -     Line 285 -label Intrnt, ref OF 1 2000, As a result of changing
111512 -     000007 below to add a "no-d" command to the 1z Supersort command
111513 -     file, the demand on Medit memory is greatly reduced because it is
111514 -     not necessary to read in a huge file that has a lot of multiple
111515 -     listings for a single file, and have macro 1201 eliminate them in
111516 -     RAM.
111517 -
111518 -
111519 -      ..
111520 -     Convert records in groups of 20 using DOSE - Process 3K Records
111521 -
111522 -     Line 360 -label Intrnt, ref OF 1 2149, created new code that works
111523 -     from two files that contain the SDS record pointers selected by
111524 -     06116. Initially tried 50, then 40 and 30, but wound up using
111525 -     blocks of 20, so that the system prepares blocks of 20 pointers to
111526 -     process using the dose command, per Morris' suggestion, ref SDS 9
111527 -     1752, and the 070405 routine, so that each session has a fresh set
111528 -     of memory space.  Did not have to use a batch file, since we can
111529 -     call 070405 to do everything as a param to opening a file in
111530 -     Medit.
111531 -     ..
111532 -     This solves the memory leaking problem so the thing will run
111533 -     continuously.  It is limited only by Medit file size which is
111534 -     about 2,000 records under the load of this process with SDS
111535 -     overhead.  Which is several years for SDS and perhaps 4 years for
111536 -     most other projects.
111537 -
111538 -         [On 980325 had step 20 down to 5, to get this to run really
111539 -         continuously in W95, ref SDS 19 1368]
111540 -
111541 -         [On 010328 memory problem occurred again, so changed the
111542 -         entire design. ref SDS 21 LP6J
111543 -
111544 -     The first step is to create a file with 30 pointers, then call
111545 -     070405 with DOSE to process it, then close that session and come
111546 -     back to 06116 to create another group of 30 pointers until the
111547 -     entire list is processed.
111548 -
111549 -     The algorithm is a little complex, because we have to build
111550 -     multiple 1y lists for the 070405 routine to process.  I did not
111551 -     want to use Cut & Paste, per, ref SDS 7 6360, so we use a series
111552 -     of deleting stuff from one file, saving it to another and reading
111553 -     it in and keeping the stuff that was deleted from the prior file,
111554 -     etc.
111555 -
111556 -         [On 010328 eliminated this entire process. ref SDS 21 LP6J
111557 -
111558 -     Line 30 -label 06116L, ref OF 3 1889, had to add a little code in
111559 -     070405 to process 1y files, which is only needed when this 06116
111560 -     operation runs.
111561 -
111562 -
111563 -      ..
111564 -     Less than 20 Records Processed by 06116 and 070405 directly
111565 -
111566 -     Line 360 -label lpInt1, ref OF 1 3166, Initially, I tried using
111567 -     30, but had to change it to 20, so that if the 1y pointer output
111568 -     file has less than 20 records, or after processing in groups of 20
111569 -     reaches less than 20 records, then the code reverts to direct
111570 -     processing rather than using the dose step.
111571 -
111572 -         [On 971209 an "update" feature was added. ref SDS 1 1428.  It
111573 -         uses the direct call to 070405, rather than using DOSE,
111574 -         because generally, there less than 30 records to call, and it
111575 -         takes a lot of overheead I don't want to put in 070405.]
111576 -
111577 -         [On 980325 had step 20 down to 5, to get this to run really
111578 -         continuously in W95, ref SDS 19 1368]
111579 -
111580 -
111581 -
1116 -

SUBJECTS
Concept Objectives
Convert SDS Record to HTML for Local Viewing
SDS Record Web Page, 070405
SDS Record Body Elements

1507 -
150701 -  ..
150702 - Numbering Executive Summary, Action Items
150703 -
150704 - Decided to create a conventional numbering method for the
150705 - Executive Summary and Action Items in SDS memos produced by 070405.
150706 -
150707 -
150708 -
150709 -  ..
150710 - 0704050 - Format SDS Record as Memo and Add HTML
150711 -
150712 - Did major re-write of HTML.  Now 070405 is the only processor that
150713 - converts SDS records into a memo and adds HTML.  It creates a memo
150714 - format, enters HTML and displays the record for identification as an
150715 - attachment to a formal correspondence.  It is also used to save the
150716 - current record to HTML data base, and it is called as a subroutine by
150717 - 06116 to convert specified groups of SDS records and save them to the
150718 - HTML data base.
150719 -
150720 -      [See explanation of major elements in 070405 in record for
150721 -      971213, when a lot of this work was completed. ref SDS 15 2932]
150722 - ..
150723 - As a result, I removed this code from 06116, and deleted 070407
150724 - which was created on 971206, ref SDS 10 3557, since 070405 now does
150725 - this function.
150726 -
150727 - Changes to 070405 were made on 971210 and 971211 because what we had
150728 - did not provide a consistent memo format for SDS records, nor display
150729 - consistently well on the Web browser, and we could not produce more
150730 - than 50 or so HTML records at a time under the old 06116 design.
150731 - ..
150732 - A major problem using HTML for Web browsers is line spacing and
150733 - outline structure.  Formatting HTML with conventional tools results in
150734 - a huge amount of code that seems impractical to replicate in the SDS
150735 - environment.
150736 -
150737 - Bill DeHart loaned me his CD for Adobe Pagemill on 971210. ref SDS 14
150738 - 1650  I tried using it create a format that was acceptable, then
150739 - conform coding to achieve that result.  It seemed, however, to produce
150740 - different results, it seemed to use different coding for the same
150741 - result and the coding seemed redundant.
150742 -
150743 - I expect what we have is tenuous.  I have tried it with both Netscape
150744 - 4.04 and Explorer 3.0.  So far it seems okay, but that does not mean
150745 - that someone else's Web browser can read it.
150746 -
150747 -
150748 -      ..
150749 -     Two Formats:  "Memo" and "Data Base"
150750 -
150751 -     Made a basic memo format that shows from whom the memo is issued
150752 -     and the date.
150753 -
150754 -     Line 830 -label 02, ref OF 3 2817, Added code to omit the From,
150755 -     date and memo ID fields for regular diary records, since they are
150756 -     not "from" anyone, and the date of the diary is the date of the
150757 -     record.
150758 -
150759 -     Added and "align" left for rec description line, to compensate for
150760 -     jumping the "From" code that has the align left command.
150761 -
150762 -      ..
150763 -     Conventional Numbering for Executive Summary
150764 -
150765 -     Line 1640 -label eol3, ref OF 3 3145, added call to new subroutine
150766 -     that removes SDS line numbers and replaces with conventional
150767 -     numbers.
150768 -
150769 -     Line 2290 -label convNum, ref OF 3 5969, this is the actual code.
150770 -
150771 -      ..
150772 -     Avoid Linking Exec Summary Line to Another Exec Sum Line
150773 -
150774 -     Encountered today an example of a link from one line in the
150775 -     Executive Summary to another line also in the Executive Summary.
150776 -
150777 -     Line 2180 -label lnk1 added code to avoid this problem which can
150778 -     occur, where a 4 place line number has a comparable structure to
150779 -     the first 4 places of a 6 place number, per explanation above for
150780 -     06116. ref SDS 0 4387
150781 -
150782 -         This effort failed because it puts a space in the anchor
150783 -         string, and HTML crashes with a space in the anchor.
150784 -     ..
150785 -     Line 2250 -label eol3, ref OF 3 6570, tried adjusting the
150786 -     search string to find the location for the anchor.
150787 -
150788 -
150789 -      ..
150790 -     Sub-level Outline Support in Summary
150791 -
150792 -     Change this so it can handle an outline structure, when there are
150793 -     subheadings under a main heading.  The entry will appear in the
150794 -     Executive Summary without a line number in order to convey it is
150795 -     part of the main item above it.
150796 -
150797 -     Line 2240 -label eol3, ref OF 3 3145, modified code, to set a flag
150798 -     when it encounters indented text.
150799 -
150800 -     Line 2310 -label convNum, ref OF 3 5969, modified this to avoid
150801 -     incrementing the line number and simply remove the SDS line
150802 -     numbers, when the indented text flag is on.
150803 -
150804 -
150805 -      ..
150806 -     "Summary/Objective" on Primary Outline Level in Exec Summary
150807 -     ------------------------------------------------------------
150808 -     Line 220 -label lpx12, ref OF 3 1084, want to omit level 4
150809 -     headings unless specifically highlighted because these lines are
150810 -     normally for time and discussion.  They usually have an underlined
150811 -     heading anyway so they are somewhat redundant to the Summary.
150812 -
150813 -     Want Summry/Objective in Executive Summary, because it contains a
150814 -     summary of the overal record.
150815 -
150816 -      ..
150817 -     Highlight Action Items and Link Anchors
150818 -
150819 -     Links to Executive Summary entries are visible because they are
150820 -     highlighted in red.
150821 -
150822 -     Links to Action Items are highlighted in green, and this is not
150823 -     striking enough in the Web browser.  Green is faded.
150824 -
150825 -     Tried changing color to yellow and that is unreadable with our
150826 -     white background.
150827 -
150828 -     Line 1680 -label eol3, ref OF 3 7933, tried eliminating color for
150829 -     Action Items lines and use only bold and underline.  This seems
150830 -     to work better.
150831 -
150832 -     Line 1660 -label eol3, ref OF 3 3051, decided to avoid including
150833 -     SDS line number field in the highlighted text for a link target.
150834 -
150835 -
150836 -      ..
150837 -     Avoid Link ID Across SDS Line Number Field
150838 -     Line Number Filed, Avoid Linking Split Citations
150839 -
150840 -     Until we figure out how to turn off the underline feature for
150841 -     HTML link sources, I want to limit the spec to the initial line,
150842 -     when a citation is split between lines.
150843 -
150844 -     Line 1520 -label SDSr, ref OF 3 4401, trying debug
150845 -
150846 -     Line 1820 -label nHTml, ref OF 3 3494, this code loads a fr
150847 -     command to position the cursor for entering the anchor closing
150848 -     HTML code.  Currently, if the anchor spec is not found on the
150849 -     current line, then the next line is processed, because that is
150850 -     where it must be.
150851 -
150852 -     Today, I disconnected this double search, so if the anchor is not
150853 -     found on the current line, then the closing anchor HTML code will
150854 -     be put at the end of the current line, so that we do not have a
150855 -     dumb looking situation of a long line from the line number field
150856 -     to the end of the citation that is often indented.
150857 -
150858 -     If we later find a way to turn off the link underline feature,
150859 -     then I can restore this code that is being unplugged.
150860 -
150861 -
150862 -      ..
150863 -     Remove Heading Underlines
150864 -
150865 -     Line 690 -label rmVSH, ref OF 3 2821, trying to eliminate dashed
150866 -     underline for major headings, since they show up well enough with
150867 -     the red characters.
150868 -
150869 -      ..
150870 -     Link Anchors - Make Distinctive
150871 -
150872 -     Line 1520 -label carot, ref OF 3 4273, make link anchors itallics,
150873 -     so the line is readily evident.
150874 -
150875 -
150876 -      ..
150877 -     Citations on a Heading Line
150878 -
150879 -     The record of the meeting with Dave Vannier on 950927 has a
150880 -     problem at ref SDS 2 4008,
150881 -
150882 -     Line 720 -label lp1, ref OF 3 1653, Testing debug
150883 -
150884 -     The cause is a citation at the end of a line that is also a
150885 -     summary heading line.  This causes the code looking for the end of
150886 -     the citation to get lost under processing at ref OF 3 0050, gets
150887 -     confused in not being able to find the end of the citation string.
150888 -
150889 -     Line 1040 -label cites, ref OF 3 0050, Testing debug
150890 -
150891 -     Line 1070 -label ref2, ref OF 3 3775, adding code to try an jump
150892 -     over HTML code for the Executive Summary, if there is already
150893 -     code on this line.
150894 -
150895 -
150896 -  ..
150897 - 000007 - Changed Selection Process to "Do-Duplicate" Keys
150898 -
150899 - Follow up ref SDS 7 7296.
150900 -
150901 - Line 1380 -label noGBL, ref OF 4 2588, added option to use a different
150902 - selection spec for 06116 based on global 76 which is set to 1 for
150903 - 06116. ref OF 1 1547  This uses a "no-d" command for "no duplicate
150904 - keys" and does not requre secondary keys.
150905 -
150906 - It should improve performance considerably so that Medit does not have
150907 - read in the entire pointer file that has a not of duplicates that are
150908 - needed for subject searches, but are not needed for HTMl because it
150909 - processes an entire file.
150910 -
150911 -
150912 -
150913 -
1510 -

SUBJECTS
Contacts
Phone Suppress Personal
Cellular Phone suppress
Personal Phones in Contacts Remove from Published Records
Cell Phones in Contacts Remove from Published Records
Phones Remove from Published SDS Records on Web
Personal Names Remove All Contact References

2110 -
211001 -  ..
211002 - Contacts Suppress Personal Phone and Cellular Phone Numbers
211003 -
211004 - Line 830, ref OF 3 1092, -label lpphpn code locates symbol "\" for
211005 - personal phone numbers.  It verfies the line is a Contact, and if so,
211006 - deletes the string beyond the id symbol, \
211007 -
211008 - Did the same for "cell" which signifies a cellular phone.  The
211009 - typical way to enter this in a Contact record is...
211010 -
211011 -                      cellular or cell 415 298 3215
211012 -
211013 - It is most often appended to the direct office phone number.
211014 -
211015 - Cellular phone numbers can be omitted from SDS records by entering
211016 - the string more than two positions beyond the end of the regular
211017 - phone, since the contact code stops if there are more than two blanks
211018 - positions.
211019 -
211020 -     [On 980315 removed links to SDS contact records from the Contact
211021 -     section of SDS records on the web. ref SDS 17 0952]
211022 -
211023 -     [On 980329 Morris suggested not publishing private phone numbers,
211024 -     which is accomplished by the code today. ref SDS 20 3675]
211025 -
211026 - Personal Contact lines are deleted, so there no representation of
211027 - people as individuals on the Internet, which is a public forum.
211028 - Business contacts should not present a problem, since SDS records
211029 - provide free advertising.
211030 -
211031 -
211032 -
211033 -
211034 -
211035 -
211036 -
211037 -
211038 -
211039 -
211040 -
211041 -
211042 -