CONTACTS 
0201 - National Science Foundation                                                                                                                                        O-00000716 0503
020101 - Ms. Sarah Nerlove; 703 306 1395 Ext 5247                                                                                                                         O-00000716 0503

SUBJECTS
NSF Proposal 9961176
Administration, Receipt, Processing
Award Schedule
Atititude Makes Resistance to Communication Metrics Emotional

1006 -
1006 -    ..
1007 - Summary/Objective
1008 -
100801 - Follow up ref SDS 46 0000, ref SDS 42 0000.
100802 -
100803 - NSF found Communication Metrics is a good idea. ref SDS 0 9435
100804 - Reviewers suggest the "need" for adding "intelligence" to management,
100805 - and research planning can be improved.  There is no comment on the
100806 - underlying goal to strengthen alphabet technology, i.e., literacy, and
100807 - no reference to "intelligence" cited in the proposal as foundational
100808 - to using technology usefully for management.  No other awards are
100809 - cited for a system of Communication Metrics, so duplication does not
100810 - seem to have been an issue.  It appears Reviewers did not access the
100811 - proposal on the Internet, as recommended by NSF staff to demonstrate
100812 - Com Metrics has been implemented and found to deliver useful
100813 - "intelligence." ref SDS 0 1764  This omission colored the review with
100814 - a false impression of exaggerated theory, ref SDS 0 2R9G, and
100815 - inability to perform due to lack of credentials, that resulted in not
100816 - making an award, because reviewers lack experience in the field of
100817 - Knowledge Management, even though they may be widely read on the
100818 - subject. ref SDS 0 0865,
100820 -  ..
100821 - Another factor may have been review by computer science, rather than
100822 - inclusion of management, behavioral science expertise on review panel.
100823 -
100824 -     [On 991221 called Sally at NSF on review. ref SDS 54 0001]
100825 -
100826 -     [On 000124 notified Roy Roebuck on results. ref SDS 57 0001]
100828 -      ..
100829 -     [On 000510 Doug Engelbart was denied NSF support on grounds that
100830 -     "knowledge" was not defined in proposal. ref SDS 62 0004
100831 -
100832 -
100834 -  ..
1009 -
1010 -
1011 - Progress
101201 -  ..
101202 - NSF Responds to Proposal for Communication Metrics Management Science
101203 -
101204 - Received ref DRT 1 0001 signed by the Division Director, Louis A.
101205 - Martin-Vega, responding to proposal 9961176 for Com Metrics, submitted
101206 - on 990616. ref SDS 27 0001, via transmittal letter, ref DIP 3 0977
101207 - which included the....
101208 -
101210 -                    ..
101211 -                   Project Summary, ref OF 11 0001
101212 -                   Table of Contents, ref OF 13 4144
101213 -
101214 -
101215 - ...based on objectives set on 990225. ref SDS 18 0788
101217 -  ..
101218 - NSF does not reference receipt on 990827 of ref DIP 14 6399 to Sally
101219 - proposing to change commercial focus of the project to Common
101220 - Administration. ref SDS 46 0001  Lack of critical analysis in the
101221 - review, indicates this added information may not have influenced the
101222 - result.
101224 -  ..
101225 - NSF missison is reported on 990404. ref SDS 21 2112
101227 -  ..
101228 - NSF cannot support the proposal for Communication Metrics, ref DRT 1
101229 - 0001, based on reports by the review panal, limited funding,
101230 - duplication and NSF program balance. ref DRT 1 2365
101231 -
101232 -     NSF web site shows no awards that duplicate the scope of the
101233 -     proposal for Com Metrics.
101235 -      ..
101236 -     [On 991221 called and talked to Sally who advised awards for the
101237 -     current peiord will not be posted to the web until next month; she
101238 -     did not cite any other projects that support Communication
101239 -     Metrics. ref SDS 54 0001]
101241 -  ..
101242 - Inquires should go to Sally. ref DRT 1 4672
101244 -  ..
101245 - The second page has general information on NSF procedures. ref DRT 1
101246 - 5280  It says....
101247 -
101248 -     1.  Approximately 200 awards were made out of 1420 proposals.
101249 -         ref DRT 1 5544
101250 -
101251 -     2.  Reviews are advisory input to NSF. ref DRT 1 1974
101252 -
101253 -         Reviewers may cite specific references in the proposals and
101254 -         offer suggestions for improvements. ref DRT 1 1974
101256 -          ..
101257 -     3.  NSF may consider factors other than the reviews. ref DRT 1
101258 -         6240
101260 -          ..
101261 -     4.  Some meritorious proposals could not be approved due to lack
101262 -         of funds. ref DRT 1 1024
101264 -          ..
101265 -     5.  Award factors are specified again. ref DRT 1 1024
101266 -
101267 -          •  Potential commercial application,
101268 -          •  Past performance commercializing SBIR/STTR results
101269 -          •  NSF program emphasis and balance.
101270 -
101271 -
101273 -  ..
101274 - Computer Science Review Panel May Be Incorrect for Com Metrics
101275 - Com Metrics Needs Review by Management, Cognitive, Computer Science
101276 -
101277 - The NSF Review Panel for...
101278 -
101279 -
101280 -                   OOSBlIT-19 Intelligent Systems
101281 -
101282 -
101283 - ...considered the proposal on Com Metrics. ref DRT 1 0024
101285 -  ..
101286 - This met the goal of introducing new ideas for improving management
101287 - and earnings set on 990225. ref SDS 18 0788
101289 -  ..
101290 - Typically, "intelligent systems" work is focused in computer science,
101291 - for artificial intelligence that is theoretical, rather than practical
101292 - application, as seen from the review of Campbell's book on 900303,
101293 - ref SDS 1 4284, and work on "mediators" at Stanford, reviewed on
101294 - 940603. ref SDS 2 0650
101296 -  ..
101297 - Communication Metrics is a practical method of improving use of the
101298 - alphabet that auguments innate human capabilities to plan, perform and
101299 - report on the work. ref OF 1 1104  An example is this record capturing
101300 - the chronology of experience with connections of cause and effect that
101301 - are segmented into chunks of context relative to objectives. This body
101302 - of "intelligence," under the definition in POIMS, ref OF 1 0582, can
101303 - then be delivered via Internet to build and maintain shared meaning
101304 - for effective collaboration on common objectives, as set out in the
101305 - Communication Metrics web site...
101306 -
101308 -                       ..
101309 -                      http://www.welchco.com
101310 -
101312 -  ..
101313 - On 990715 Sally objected to this method, ref SDS 37 0001, so this may
101314 - be an influence in the review.
101316 -  ..
101317 - Like the alphabet, SDS is a tool that leverages human thinking,
101318 - explained in POIMS ref OF 1 0858 and ref OF 1 6528   The proposal
101319 - states this expressly in the first para, ref OF 14 0001  Like the
101320 - alphabet, SDS does not "think" nor seek to accomplish intelligence for
101321 - people, but rather helps people think for themselves and generate
101322 - useful "intelligence."  Therefore, reviewing Com Metrics that applies
101323 - SDS requires experts who have investigated the nature of thinking, as
101324 - in Campbell's book, and Landauer's paper on LSA reviewed for example
101325 - on 960518. ref SDS 7 8811
101327 -  ..
101328 - Proposal Part 2, Significance of Innovation cites reliance on
101329 - cognitive science in the first paragraph, and the objective to support
101330 - Enterprise Management. ref OF 14 0001
101332 -  ..
101333 - The NSF review panel may have had people with management and
101334 - behavorial science expertise, however, that expertise is not evident
101335 - in comments submitted by NSF.  Moreover, "Intelligent systems" often
101336 - connotes computer scientists.  People familiar with management
101337 - science, connectionist theory and alphabet technology need to assess
101338 - the proposal, as discussed with Cheryl and Joe on 990614, ref SDS 24
101339 - 0866, confirmed in the letter to NSF on 990615. ref DIP 2 0001
101340 -
101341 -     [On 991221 Sally Nerlove at NSF was not aware that Com Metrics
101342 -     needs cognitive science review. ref SDS 54 5014]
101344 -      ..
101345 -     [On 000510 Doug Engelbart was denied NSF support on grounds that
101346 -     "knowledge" was not defined in proposal. ref SDS 62 0004
101348 -  ..
101349 - If NSF has a program for management science, possibly further review
101350 - is warranted based on using technology to leverage human cognition.
101352 -  ..
101353 - The SDS program and the Internet use computers.  Such experts should
101354 - have something helpful to contribute, so long as the objectives of the
101355 - proposal are recognized in evaluating the proposal.  Here that appears
101356 - not to have occurred.
101357 -
101358 -
101359 -
101360 -
101361 -
1014 -

SUBJECTS
Panel Conclusions
Willfully Blind, Fear and Denial

1204 -
120501 -  ..
120502 - NSF Found Communication Metrics Sets Important Goal
120503 -
120504 - NSF Review requires...
120505 -
120506 -     State the proposal's aim; comment on the proposal's strengths,
120507 -     weaknesses and commercial potential as discussed by the panel;
120508 -     and, suggest improvements for the research plan and or proposal
120509 -     preparation. ref DRT 1 2548
120511 -  ..
120512 - NSF Review Panel found...
120513 -
120514 -     The researcher proposes to create an internet search capability
120515 -     that allows users to assign multiple subjects to chunks of
120516 -     information and to chain information by time. The claim is that
120517 -     this would facilitate information retrieval. ref DRT 1 1260
120519 -      ..
120520 -     This is an ambitious goal but the argument for the need should be
120521 -     more clearly articulated and the proposal needs a clearly
120522 -     formulated research plan. ref DRT 1 3871
120524 -  ..
120525 - The lead reviewer states...
120526 -
120527 -     I can imagine that a highly functional system of the kind hinted
120528 -     at in the proposal could make a substantial contribution to
120529 -     knowledge management and, in a period when that phrase is
120530 -     increasingly popular in data mining systems, electronic commerce,
120531 -     and similar applications areas, that it could offer commercial
120532 -     value. But there is no evidence that what is proposed here would
120533 -     be a highly functional system. ref DRT 1 3630
120535 -  ..
120536 - Second Reviewer says...
120537 -
120538 -     Retrieval from experience is certainly desirable. ref DRT 1 2244
120539 -
120541 -  ..
120542 - Third Reviewer says...
120543 -
120544 -     The PI proposes to improve enterprise management by developing
120545 -     business metrics for improved communication.
120546 -
120547 -     The goal is certainly ambitious.  However, the text is unclear as
120548 -     to what will be done, hugely overstated in its claims, and
120549 -     entirely lacking in any documentation of these claims. ref DRT 1
120550 -     3723
120551 -
120553 -  ..
120554 - Click Links Omitted Ignorance SDS Exists Seems Like Only Theory
120555 - NSF Concluded Com Metrics is Only Theory, Due to Inexperience
120556 -
120557 - On 990614 NSF staff Joe Hennessy and Cheryl Albus recommended that
120558 - the Welch proposal provide access to the SDS record on the Internet so
120559 - that reviewers can understand ideas on "intelligence," and recognize
120560 - that Communication Metrics is not just a theory, but rather is a
120561 - practical solution to an important problem, and has been largely
120562 - implemented. ref SDS 24 5451  NSF's recommendation applies Einstein's
120563 - observation that experience is the foundation of knowledge, reported
120564 - on 991124. ref SDS 51 3066
120566 -  ..
120567 - On 990615 Joe Hennessy reported having experienced Communication
120568 - Metrics on the Internet and confirmed the prior recommendation that
120569 - this is helpful to understand the proposal. ref SDS 25 1044
120571 -  ..
120572 - Joe and Cheryl also recommended that a list of potential reviewers who
120573 - have experience with the SDS record on the Internet be submitted to
120574 - NSF for selection of a review panel. ref SDS 24 9903
120576 -  ..
120577 - On 990616 a list of reviewers was submitted, and the proposal provides
120578 - direct access to the SDS record on the Internet.
120580 -  ..
120581 - NSF review comments show failure to review the Com Metrics proposal
120582 - online, as requested in the proposal. ref OF 14 4402  Using the
120583 - printed version prevented gaining experience showing implementation of
120584 - Com Metrics that enables a "paperless office" that augments human
120585 - intelligence, causing a false conclusion that Com Metrics is only
120586 - theory, and cannot be accomplished, further mirroring the error made
120587 - by peer review at Intel on 990713. ref SDS 34 1767
120588 -
120589 -     [On 020726 researcher left SRI because of slow progress on work
120590 -     sponsored by NSF and EPA. ref SDS 69 K16J
120592 -  ..
120593 - Lead Reviewer says...
120594 -
120595 -     "... can imagine that a highly functional system of the kind
120596 -     hinted at in the proposal could make a substantial contribution to
120597 -     knowledge management and, in a period when that phrase is
120598 -     increasingly popular in data mining systems, electronic commerce,
120599 -     and similar applications areas, that it could offer commercial
120600 -     value. But there is no evidence that what is proposed here would
120601 -     be a highly functional system," ref DRT 1 3630,
120603 -  ..
120604 - The reviewer errs describing the Welch proposal as "hinting" at a
120605 - highly functional system, since the proposal expressly states the
120606 - system lifts the capacity to think, remember and communicate, and it
120607 - provides direct evidence, not hints, that the system is functional.
120609 -  ..
120610 - Aligns with review by Intel on 990713 who were ignorant of SDS because
120611 - they were shielded from the record on the Internet. ref SDS 34 2150
120612 -
120613 -      [On 010425 Morris reported SDS use cases are a "utopia" compared
120614 -      to methods other people use, which reflects NSF review that Welch
120615 -      proposal is overstated, and cannot be performed. ref SDS 67 EP7F
120617 -  ..
120618 - Struggeling to "imagine" Com Metrics would be useful rather than
120619 - commenting on experience interacting with the SDS record on the
120620 - Internet, shows the Reviewer is unaware that the system already
120621 - exists, and is therefore fully functional.
120622 -
120623 -      [On 000503 Jack Park had similar experience with NIH. ref SDS 61
120624 -      3025
120626 -       ..
120627 -      [On 000510 Doug Engelbart was denied NSF support on grounds that
120628 -      "knowledge" was not defined in proposal. ref SDS 62 0004
120630 -  ..
120631 - Reviewer's contention there is no evidence that what is proposed would
120632 - be a highly functional system is incorrect, since the existence of the
120633 - SDS record on the Internet is direct evidence that Communication
120634 - Metrics is a functional system. Second, reports by the U.S. Army Corps
120635 - of Engineers that Communication Metrics was used to save money for
120636 - taxpayers is cited three (3) times in the proposal, ref OF 14 0556 and
120637 - ref OF 14 2124, and comprises direct evidence of a highly functional
120638 - system, since ROI is 10:1.
120640 -  ..
120641 - Second reviewer says...
120642 -
120643 -     "Retrieval from experience is certainly desirable. However, in
120644 -     this system what the user can retrieve is only as good as the
120645 -     information he inputs.  This is a major flaw as most users will
120646 -     not have time to do all this input." ref DRT 1 2244
120648 -  ..
120649 - Reviewer's comment shows failure to access the proposal online, since
120650 - doing so reveals a much wider range of context is provided by SDS than
120651 - merely what is "input" by a user, as shown by this record, which is
120652 - linked to the NSF review, NSF requirements, and other sources that
120653 - have not been input by the user, but bear on the meaning and action to
120654 - be taken by the user and others.  That is the magic of "intelligence"
120655 - that adds value to information by creating "knowledge," and is
120656 - overlooked today by NSF, because they do not know the distinction
120657 - between information and knowledge, and do not have the benefit of
120658 - using SDS, but are merely passing on whether it would be helpful to
120659 - have something about which they have chosen to be ignorant by refusing
120660 - to click on a link.
120662 -  ..
120663 - Refusal by NSF reviewers to access links in the proposal aligns with
120664 - complaints on 990715 by Sally Nerlove, NSF lead for this proposal
120665 - area, ref SDS 37 0001  It, also, aligns with cultural resistance to
120666 - new methods embodied by the legend of Prometheus reviewed on 991108.
120667 - ref SDS 50 5368  While resistance to new methods should be expected
120668 - (Andy Grove notes people like to work on familiar things in familiar
120669 - ways, reviewed on 980307. ref SDS 15 3740) some accomodation should be
120670 - expected by the agency entrusted to assess new methods.
120672 -  ..
120673 - Belief that retrieval in SDS is only as good as information "input"
120674 - ignores the value of connections human intelligence adds to otherwise
120675 - improvished information, well recognized in cognitive science, see for
120676 - example review on 990303 of George Miller's early work. ref SDS 19
120677 - 2838  Previous review on 900303 made the same point. ref SDS 1 4456
120678 - Experiencing SDS online shows that many connections human intelligence
120679 - makes are in SDS records.  So much so, that some people are
120680 - overwhelmed, as reported on 980712. ref SDS 16 6804  Sally may have
120681 - had this experience on 990715, ref SDS 37 0001, leading to the NSF's
120682 - unfavorable review today.
120684 -  ..
120685 - Moreover, experiencing "intelligence" on the Internet exposes the flaw
120686 - in the hunter/gatherer view that executives, lawyers, engineers,
120687 - doctors, mechanics and other knowledge workers, do not have time to
120688 - invest intellectual capital, since in fact reliance on accurate
120689 - experience is the only method humans have to "save time," as reviewed
120690 - on 950426. ref SDS 3 4404
120692 -  ..
120693 - The third review says...
120694 -
120695 -     "The goal [of business metrics for communication] is
120696 -     certainly ambitious. However, the text is unclear as to
120697 -     what will be done, hugely overstated in its claims, and
120698 -     entirely lacking in any documentation of these claims.
120699 -     ref DRT 1 3723
120701 -  ..
120702 - Concluding the proposal is unclear as to what will be done conflicts
120703 - with the first two reviews, and indicates cursory consideration, per
120704 - above, ref SDS 0 0940, as occurred at Intel on 990713, where three
120705 - managers overlooked the problem of Information Overload cited in the
120706 - first paragraph of the proposal. ref SDS 34 9900
120707 -
120708 -     [On 000716 Professor Joseph Ransdell reported Com Metrics is
120709 -     ambitious because it provides a theory of management that has
120710 -     eluded disocvery since the 17th century, and he urged not to get
120711 -     discouraged by slow acceptance. ref SDS 64 7838
120713 -  ..
120714 - The feeling that a claim of delivering "intelligence" on the Internet
120715 - is "overstated," shows failure to experience Communication Metrics on
120716 - the Internet.  The existence of the implementation shows the ability
120717 - to perform as noted by Joe Hennessy on 990614. ref SDS 25 1044
120719 -  ..
120720 - The implication is that very capable people are numbed and frozen by
120721 - the prospect of adding intelligence to management, and simply cease to
120722 - think rigorously and honestly when confronted with a proposal to
120723 - improve their work, as observed by Andy Grove in his book "Only the
120724 - Paranoid Survive," reviewed on 980307. ref SDS 15 2044
120725 -
120726 -
120728 -  ..
120729 - NSF Finds Need for "Intelligence" Not Established by Proposal
120730 -
120731 - NSF Review requires...
120732 -
120733 -     State the proposal's aim; comment on the proposal's strengths,
120734 -     weaknesses and commercial potential as discussed by the panel;
120735 -     and, suggest improvements for the research plan and or proposal
120736 -     preparation. ref DRT 1 2548
120738 -  ..
120739 - Panel found no systematic argument on the need for Com Metrics...
120740 -
120741 -     The researcher proposes to create an internet search capability
120742 -     that allows users to assign multiple subjects to chunks of
120743 -     information and to chain information by time. The claim is that
120744 -     this would facilitate information retrieval. While the researcher
120745 -     addresses the need for this in very general claims, there is no
120746 -     systematic argument for this need. ref DRT 1 1260
120748 -  ..
120749 - Lead Review says...
120750 -
120751 -     The proposal references a critical information management need -
120752 -     essentially, organizing time and information through software that
120753 -     supports intelligent, shared linkages. It does so without defining
120754 -     in any systematic way the nature of that need,... ref DRT 1 8700
120756 -  ..
120757 - Analysis...
120758 -
120759 -     1.  NSF Review does not align well with proposal objectives to
120760 -         deliver "intelligence" on the Internet.
120761 -             ..
120762 -         a.  The project summary says...
120763 -
120764 -             Research objectives are (1) to create a search capability
120765 -             on the Internet that assembles chains of chronology by
120766 -             subject content to show causation over specified time
120767 -             spans; (2) a means is needed to assign multiple subjects
120768 -             to chunks of information based on personal and
120769 -             organizational objectives that control how information is
120770 -             assembled for discovering causation, and to share
120771 -             knowledge structures that build and maintain shared
120772 -             meaning; (3) guidelines for clear, concise, complete
120773 -             communication support effective use of the Internet.
120774 -             ref OF 11 6800
120776 -              ..
120777 -         b.  Significance of Innovation says...
120778 -
120779 -             The project will research Communication Metrics on the
120780 -             Internet to deliver intelligence ... the human brain
120781 -             connects information into chains of cause and effect
120782 -             through a process called "intelligence." This new project
120783 -             proposes research on using technology that leverages
120784 -             innate intelligence by integrating time and information,
120785 -             which produces "knowledge,"  This adds value to human
120786 -             experience... [supports] paperless office. ref OF 14 0001
120788 -              ..
120789 -             Another major innovation in SDS are tools and procedures
120790 -             for organizing information according to an organic subject
120791 -             structure to facilitate retrieval. ref OF 14 3922
120793 -  ..
120794 - Accordingly, NSF review can be strengthened by offering a richer
120795 - explanation based on the language in the proposal.
120797 -  ..
120798 - An example to improve NSF review is as follows...
120799 -
120800 -       The Welch proposal claims SDS technology facilitates connecting
120801 -       information into chronologies of cause and effect segmented by
120802 -       subject content, as an extension of alphabet technology, for
120803 -       capturing and managing daily experience in relation to personal
120804 -       and organizational objectives.  The methodology is summarized by
120805 -       the notion of integrating time and information.  Documents and
120806 -       Contacts are integrated into the experiential record for context
120807 -       management.  Chronologies of cause and effect are claimed to
120808 -       provide traceability to original sources.  This method reflects
120809 -       root cause analysis, case studies, lessons learned, history, or
120810 -       more simply what is commonly called "knowledge."
120812 -        ..
120813 -       "Intelligence" is defined in the proposal as a continual process
120814 -       of plan, perform, report to organize, align, analyse, develop
120815 -       summary connected to detail, provide and obtain feedback.
120816 -       Alignment identifies deviations from facts, objectives and
120817 -       requirements that permit adjusting course to avoid mistakes.
120819 -        ..
120820 -       The proposal claims SDS organic subject structures and the
120821 -       entire program environment permit flexible retrieval of
120822 -       information, reflective of human "intelligence."
120824 -        ..
120825 -       The SDS experiential record has been ported to the Internet as a
120826 -       means to deliver business intelligence.  The U.S. Army Corps of
120827 -       Engineers applied Com Metrics and reports the methodology saves
120828 -       money. The current proposal is to port the SDS search capability
120829 -       to the Internet for enhancing access to "intelligence" on the
120830 -       Internet, as defined in the proposal.
120832 -  ..
120833 - NSF guidelines suggest but do not require alignment of review with
120834 - proposals. ref DRT 1 1974  Limited time, and lack of experience using
120835 - alignment may have hampered NSF review, which cascaded into further
120836 - error.
120837 -
120838 -     1.  Panel's failure to cite the organizing methodology for
120839 -         information, and guidelines for applying "intelligence" on the
120840 -         Internet prevented understanding value added to exsiting
120841 -         methods.  For example, on 990715 Sally reported not having
120842 -         time to review the record online, ref SDS 37 0001, which
120843 -         reflects need for Judicious Review processes developed on
120844 -         990419, ref SDS 22 2320, and which is part of the proposal
120845 -         entirely overlooked in the NSF review. ref OF 15 0877
120847 -          ..
120848 -     2.  The conclusion that the proposal lacks a "systematic argument"
120849 -         for the need to add "intelligence" to management by providing
120850 -         automated integration of time and information is incorrect.
120851 -
120852 -         a.  First, the life experience of anyone involved with
120853 -             knowledge work, such as a reviewer, manager, scientist,
120854 -             doctor, mechanic, or engineer, attest to the need for
120855 -             "intelligence," as defined in the proposal. ref OF 14 0001
120857 -              ..
120858 -             This view was restrained by requiring access to POIMS on
120859 -             the Internet linked in the proposal for a full definition
120860 -             of "intelligence."  However, this was a deliberate design
120861 -             in the proposal aimed at providing experience with new
120862 -             ideas in order to facilitate understanding, as recommended
120863 -             by NSF staff on 990615. ref SDS 25 1044
120865 -              ..
120866 -             Part 2, Significance of the Innovation says... The nation
120867 -             will benefit from a direct, practical means to solve
120868 -             information overload, that otherwise overwhelms human span
120869 -             of attention, causing daily bumbling, mistakes, loss,
120870 -             crisis, conflict and disaster in public and commercial
120871 -             life, described by Henry Kissinger as an "Alice in
120872 -             Wonderland" environment. The explosion of communication in
120873 -             all forms adds urgency to this project. ref OF 14 0001
120875 -              ..
120876 -             The need to solve information overload is overwhelming. It
120877 -             is not an open question.
120879 -              ..
120880 -             The national initiative to cut medical mistakes reported
120881 -             on 991207, ref SDS 52 0001, and NASA's continual problems
120882 -             e.g., reported on 991001, ref SDS 49 0001, give visibility
120883 -             to a crushing problem every manager and every knowledge
120884 -             worker experiences every day.  These examples are not
120885 -             cited in the proposal, but evidence cultural awareness of
120886 -             pandemic bumbling caused by lack of "intellignece," as
120887 -             defined in the proposal.  Only the willfully blind are
120888 -             ignorant of this need.
120890 -              ..
120891 -         b.  The proposal cites the report by the U.S. Army Corps of
120892 -             Engineers on Oct 7, 1997 that "subjects" overlooked by
120893 -             managers due to limited span of attention cause rework
120894 -             that increase cost and delay, which is solved by
120895 -             Communication Metrics, using SDS organic subject
120896 -             structures. ref OF 14 0556
120898 -              ..
120899 -             Need to improve conventional subject and keyword methods
120900 -             are cited in the report. ref OF 14 1650
120902 -              ..
120903 -             ROI of 10:1 is cited as justification for porting the SDS
120904 -             organic subject structure to the Internet. ref OF 14 2124
120905 -
120906 -
120907 -
1210 -

SUBJECTS
Decision Support Systems
Bayseian Belief Networks
KM Dilemma
Knowledge Management Problem

2106 -
210701 -  ..
210702 - Knowledge Management Delimma Undefined by NSF
210703 - NSF Found Research Plan Defective, Principal Investigator Lacks Skill
210704 -
210705 - Panel concurs with the proposal that experts need to be interviewed,
210706 - but maintains this should be submitted in the proposal rather than
210707 - developed in research...
210708 -
210709 -     The researcher proposes to interview computer scientists as well
210710 -     as others in knowledge classification and formulate this into a
210711 -     phase two plan. While this is work that needs to be done, it would
210712 -     make a stronger proposal to have completed this prior to
210713 -     submitting the SBIR proposal. ref DRT 1 1118
210715 -  ..
210716 - Lead Reviewer comments...
210717 -
210718 -     ...the proposal presents no real plan either for analyzing the
210719 -     knowledge management dilemma... or for developing technologies
210720 -     and methods to respond,... it shows little awareness of research
210721 -     and practice in the several fields the proposal touches,
210722 -     including computational linguistics, search and retrieval
210723 -     systems, Bayesian belief networks, clustering algorithms, other
210724 -     neural network technologies, etc. ref DRT 1 6120
210725 -
210726 -         [...see below for more "computational linguistics."
210727 -         ref SDS 0 XN3M
210729 -          ..
210730 -         [On 000110 letter from Cornell University says Computational
210731 -         Linguistics does not support Com Metrics. ref SDS 56 0001]
210733 -          ..
210734 -         [On 000106 reviewed knowledge management dilemma with Morris
210735 -         at Intel. ref SDS 55 5120
210737 -          ..
210738 -         [On 000510 Doug Engelbart was denied NSF support on grounds
210739 -         that "knowledge" was not defined in proposal. ref SDS 62 0004
210741 -          ..
210742 -         [On 011222 "paraphrasing" presents overwhelming KM dilemma
210743 -         that SDS solves. ref SDS 68 XT4K
210745 -          ..
210746 -         [On 060211 Open Iris new software release by SRI that provides
210747 -         clustering support that may implement solutions proposed today
210748 -         by NSF for intelligence support of Communication Metrics.
210749 -         ref SDS 70 QW6O
210751 -      ..
210752 -     The principal investigator... does not show evidence of having the
210753 -     intellectual and experiential tools required to approach those
210754 -     issues with anything like sufficient polish or rigor.  His reach
210755 -     exceeds his grasp. ref DRT 1 5822
210756 -
210757 -         [On 991221 Sally Nerlove at NSF reported that Welch is a
210758 -         Diamond in the Rough. ref SDS 54 0001]
210760 -  ..
210761 - Analysis...
210762 -
210763 -     1.  Asking for research to be included in the proposal puts the
210764 -         "cart before the horse," as stated in Part 2 of the proposal.
210765 -         ref OF 14 3740  This was discussed with Sally on 990629.
210766 -         ref SDS 28 0886
210768 -          ..
210769 -     2.  The proposal states that SDS solves the knowledge management
210770 -         dilemma by automated integration of time and information,
210771 -         ref OF 14 7802, which improves alphabet technology, and notes
210772 -         that part of SDS has been ported to the Internet where the
210773 -         implementation can be observed, which thereon establishes
210774 -         credentials to perform.
210776 -            ..
210777 -           [On 011222 "paraphrasing" presents overwhelming KM dilemma
210778 -           that SDS solves. ref SDS 68 XT4K
210780 -          ..
210781 -         SDS on the Internet demonstrates success accomplishing a level
210782 -         of tool support for "intelligence" that nobody else has
210783 -         accomplished using methods cited in the review. ref DRT 1 6120
210784 -         NSF reviewers cite no applications that support Enterprise
210785 -         Management with skills and methods claimed to be needed for
210786 -         solving the knowledge management delimma, as grounds for
210787 -         criticizing the Welch proposal. ref SDS 0 0844
210789 -          ..
210790 -         NSF erred in failing to click on a link to discover that
210791 -         Communication Metrics is not simply a theory, but that most of
210792 -         it already exists, and thereon demonstrates ability to perform
210793 -         beyond the capability of other grantees who have been funded,
210794 -         but have not produced useful knowledge capability.
210796 -          ..
210797 -         NSF further erred applying artificial intelligence techniques
210798 -         to evaluate Com Metrics.
210800 -          ..
210801 -         Bayseian belief networks are graphical representations of
210802 -         probabilistic relationships for reasoning under uncertainty
210803 -         using mathematical modeling that apply expert opinion about
210804 -         the dependencies between variables. ref OF 6 2405  BBNs have
210805 -         been used for medical diagnosis and analysis of mechanical
210806 -         failures, ref OF 6 7144, for safety litigation, military
210807 -         vehicle and software reliablity. ref OF 6 1760  Help systems
210808 -         in Microsoft Office; also the ‘intelligent’ printer
210809 -         fault diagnostic system which you can run when you log onto
210810 -         Microsoft’s web site use BBN to resolve printer problems.
210811 -         ref OF 6 7144
210813 -          ..
210814 -         These applications do not align with the Welch proposal to use
210815 -         technology for aiding human thinking by enhancing the alphabet
210816 -         for adding "intelligence" to management.
210817 -
210818 -             [On 000106 discussed SDS for computer aided thinking.
210819 -             ref SDS 55 1316]
210821 -              ..
210822 -             [On 010217 received question from Benham Grey with KM
210823 -             section of Brint, asking about correlation between POIMS
210824 -             technology explained on Welchco.com, and AI capability
210825 -             using Bayesian statistics and neural networks to provide a
210826 -             super search engine based extracting patterns. ref SDS 66
210827 -             5T8K
210829 -          ..
210830 -         The U.S. Air Force used statistical modeling based on expert
210831 -         opinion in hopes of preventing management information from
210832 -         degrading toward entropy over time, reported on 970707.
210833 -         ref SDS 12 0108  This method failed.  On 971008 the U.S. Army
210834 -         Corps of Engineers reported Communication Metrics succeeded.
210835 -         ref SDS 13 2710  By what reasoning can a method that has not
210836 -         worked be the basis to criticize a method that has been found
210837 -         to save money?
210839 -          ..
210840 -     3.  The panel's determination that the research plan is weak
210841 -         aligns with Mark Haselkorn's comment on 990729. ref SDS 43
210842 -         0001
210844 -          ..
210845 -         Weak research planning is not dispositive of whether the
210846 -         project can succeed based on two facts...
210847 -
210848 -         a.  Knowledge management is frontier, cutting edge science.
210849 -             Nobody knows what it means.  Most breakthroughs do not
210850 -             occur in the lab, but in the field of life where chance
210851 -             arrangements augment capability and subsequent experience
210852 -             leads to scientific theories that explain augmentation, as
210853 -             set out by Dr. Thomas K. Landauer in his book...
210854 -
210856 -                        ..
210857 -                       The Trouble with Computers
210858 -
210859 -
210860 -             ...reviewed on 950710. ref SDS 4 8303
210862 -              ..
210863 -             Intial research for knowledge management entails rough
210864 -             approximations of conventional practice in order to form
210865 -             the foundation for rigorous work.
210867 -              ..
210868 -         b.  SDS already exists, proving the theory of POIMS technology
210869 -             for augmenting human intelligence. ref OF 1 0561 However,
210870 -             weak the research plan, this foundation is far ahead of
210871 -             anything NSF might fund for this purpose.  Funding is to
210872 -             expand and sharpen subject identification, and make a
210873 -             stronger implementation on the Internet, so it is easier
210874 -             for more people to use, and therefore more commercially
210875 -             viable.  That is the objective of the NSF program for
210876 -             SBIR, i.e., to make innovation profitable.
210878 -              ..
210879 -             We want to bridge the gap between technical feasiblity, of
210880 -             POIMS technology, established by SDS, and commercial
210881 -             viability.
210882 -
210883 -
210884 -
210885 -
210886 -
210887 -
210888 -
2109 -

SUBJECTS
Decision Support Systems
Computational Linguistics
Speech Recognition
Bayesian Decision Support
Bayesian Statistics Neural Networks Search Engine Extracts Patterns

3007 -
300801 -          ..
300802 -     4.  Computational Linguistics combines humanities, natural and
300803 -         behavioural sciences, and engineering to support artificial
300804 -         intelligence (AI), cognitive science, and speech specialists
300805 -         with design and analysis of natural language processing
300806 -         systems.  Language pattern matching applications include
300807 -         grammar checkers for wordprocessing, speech and handwritting
300808 -         recognition, and language translation. ref OF 5 3994
300810 -          ..
300811 -         NSF ideas on using computational linguistics may be related to
300812 -         lead review discussion of Bayesian belief networks, per above.
300813 -         ref SDS 0 0844
300815 -          ..
300816 -         On 980226 IBM announced progress developing "data mining"
300817 -         using language pattern matching, which may be based on
300818 -         computational linguistics cited by NSF today. ref SDS 14 1247
300820 -            ..
300821 -           [On 000110 letter from Cornell University says Computational
300822 -           Linguistics does not support Com Metrics. ref SDS 56 0001]
300824 -            ..
300825 -           [On 000314 cited language pattern matching to support
300826 -           alignment, as possible research with LANL. ref SDS 58 2838
300828 -            ..
300829 -           [On 000424 Sandy Klausen suggests this is purpose of DKR
300830 -           project. ref SDS 59 4212
300832 -            ..
300833 -           [On 000428 Henry Liu may be able to support this task.
300834 -           ref SDS 60 5358
300836 -            ..
300837 -           [On 000503 Jack Park wants to pursue this task. ref SDS 61
300838 -           6860
300840 -            ..
300841 -           [On 000623 Jack proposes engine to map subjects. ref SDS 63
300842 -           2915
300844 -            ..
300845 -           [On 001025 Paul Fernhout explains challenge of identifying
300846 -           knowledge representation well enough to accomplish useful
300847 -           automated organization of information. ref SDS 65 LR6N
300849 -            ..
300850 -           [On 010217 received question from Benham Grey asking about
300851 -           correlation between POIMS technology explained on
300852 -           Welchco.com, and AI capability using Bayesian statistics and
300853 -           neural networks to provide a super search engine based
300854 -           extracting patterns. ref SDS 66 5T8K
300856 -            ..
300857 -           [On 060211 Open Iris new software release by SRI that
300858 -           provides clustering support that may implement solutions
300859 -           proposed today by NSF for intelligence support of
300860 -           Communication Metrics. ref SDS 70 QW6O
300862 -          ..
300863 -         Pyscholinguistics uses mathamatical modeling to examine
300864 -         cognitive processes of language. ref OF 5 5609
300866 -          ..
300867 -         On 960518 reviewed Doctor Thomas K. Landauer's mathamatical
300868 -         model for Latent Semantic Analysis (LSA) uses computational
300869 -         linguistics to analyse learning processes that show word
300870 -         meanings constantly change in the mind. ref SDS 7 3734
300871 -         Computational linguistics offers no solution.  The Welch
300872 -         proposal explains a method of Communication Metrics for
300873 -         maintaining alignment to prevent meaning drift. ref OF 14 5561
300875 -          ..
300876 -         POIMS and the New World Order... papers define "intelligence"
300877 -         and explain how technology can do something useful to help
300878 -         people and organizations perform knowledge work, cited in the
300879 -         proposal. ref OF 14 1148
300881 -          ..
300882 -         This record supports continued effort.
300884 -  ..
300885 - NSF Practice Justifies Support for Communication Metrics
300886 -
300887 -         Review shows NSF has made awards for information research,
300888 -         which have not advanced knowledge management, because they do
300889 -         not define "knowledge" and "intelligence" in a way that can be
300890 -         aided by technology, which has been accomplished by POIMS.
300891 -         ref OF 1 0561
300893 -          ..
300894 -         None of the abstracts show awareness of how to improve
300895 -         communication, management and earnings; none mention the need
300896 -         to solve limited span of attention, nor to support alignment
300897 -         to avoid meaning drift, explained in POIMS. ref OF 1 8774
300899 -          ..
300900 -         NSF's web resources
300901 -
300902 -
300903 -                    http://www.nsf.gov/verity/srchawd.htm
300905 -  ..
300906 -         ...show field search...
300907 -
300908 -
300909 -                    http://www.nsf.gov/verity/srchawdf.htm
300911 -  ..
300912 -         ...and the list at....
300913 -
300914 -
300915 -                   http://www.nsf.gov/cgi-bin/verity/srchawdf
300916 -
300917 -
300918 -           (access is very, very slow.)
300920 -          ..
300921 -         NSF approved award 9216848 on July 1, 1998 for...
300922 -
300923 -              A Scientific Group Communications & Collaborative Testbed
300924 -              for Upper Atmospheric Research...
300925 -
300926 -
300927 -                  http://www.nsf.gov/cgi-bin/showaward?award=9216848
300929 -          ..
300930 -         The abstract says this award is for...
300931 -
300932 -              ... a multidisciplinary effort linking research in
300933 -              computer science, behavioral science, and upper
300934 -              atmospheric and space science to build a prototype system
300935 -              for a distributed but shared working environment: the
300936 -              vision of a collaboratory.
300938 -               ..
300939 -              This research will [study] collaboration tools... The
300940 -              Sondre Stromfjord researchers expect to obtain greater
300941 -              efficiency in joint experimental operations, analysis,
300942 -              and discovery by guiding the requirements for the
300943 -              collaboration environment which they will utilize to
300944 -              support their research.
300946 -          ..
300947 -         This proposal is silent on the need to integrate time and
300948 -         information, which is the only way to grow new knowledge.
300949 -         Further, collaboration requires building and maintaining
300950 -         shared meaning. Project 9216848 is silent on solving meaning
300951 -         drift.
300953 -          ..
300954 -         NSF approved award 9812755 for a project that expires on
300955 -         August 31, 2000 for...
300956 -              ..
300957 -              Enhancing Robustness of Information through
300958 -              Distributed Adaptive Coordination
300959 -
300961 -         ..
300962 -        http://www.nsf.gov/search97cgi/vtopic?action=View&VdkVgwKey=http%3A%2F%2Fwww%2Einteract%2Ensf%2Egov%2Fcise%2Fabst%2Ensf%2FPages%2FC521F7713038AE55852567D700693B74&DocOffset=13&DocsFound=182110&SearchUrl=http%3A%2F%2Fwww%2Ensf%2Egov%2Fsearch97cgi%2Fvtopic%3FQueryText%3D%26ResultStart%3D11%26ResultCount%3D10&
300964 -          ..
300965 -         Abstract says this award is for...
300966 -
300967 -              ...[creating] software for complex and large-scale
300968 -              information systems... [Development entails] potentially
300969 -              hundreds of intelligent real-time agents.  Robust
300970 -              information systems will adapt... in response to changes
300971 -              in system goals, ... requirements... A multi-agent <8232
300972 -              architecture appropriate for such systems is being
300973 -              developed ... for... scheduling, ... coordination, ...
300974 -              design, ... inappropriate behavior... and on-line <1575
300975 -              learning to improve the performance... The impact of this
300976 -              work is to provide software, techniques and empirical
300977 -              studies that will facilitate the construction of the next
300978 -              generation of robust information systems.
300979 -              http://dis.cs.umass.edu/
300981 -          ..
300982 -         This program sounds similar to Stanford's work on mediators,
300983 -         using "intelligent" agents, reviewed on 940603, ref SDS 2 9861
300984 -         which is cited in POIMS.
300986 -          ..
300987 -         NSF approved award 9703470 for a project that expires on
300988 -         July 31, 2000 for...
300989 -
300990 -              CISE Research Infrastructure: A Next Generation Computing
300991 -              and Communications Substrate
300993 -         ..
300994 -        http://www.nsf.gov/search97cgi/vtopic?action=View&VdkVgwKey=http%2F%2Fwww%2Einteract%2Ensf%2Egov%2Fcise%2Fabst%2Ensf%2FPages%2FC5035D14D95C2094852567D7006868DB&DocOffset=18&DocsFound=182110&SearchUrl=http%3A%2F%2Fwww%2Ensf%2Egov%2Fsearch97cgi%2Fvtopic%3FQueryText%3D%26ResultStart%3D11%26ResultStart%3D11%26ResultCount%3D10&
300996 -          ..
300997 -         Abstract says this award is for...
300998 -
300999 -              ...[developing] a new computing and communications
301000 -              substrate that provides "access to anything, anytime,
301001 -              anywhere". To achieve this goal Cornell researchers plan
301002 -              to: (1) do fundamental new research, (2) develop
301003 -              integrated systems to demonstrate the power of their
301004 -              results, and (3) test these systems. In order to develop
301005 -              the proposed systems, it is critical to test them with
301006 -              real systems and real users.  These researchers plan to
301007 -              use their teaching activities as a ready source of users
301008 -              to test their systems, as well as using the systems to
301009 -              disseminate their research results and to provide
301010 -              experimental information services. Cornell University is
301011 -              moving towards a campus-wide hybrid ATM and switched
301012 -              ethernet communications network, which will eventually
301013 -              enable them to test their systems with campus-wide
301014 -              network of 20,000 users. They also expect to release
301015 -              their systems for use by other academic institutions and
301016 -              businesses.
301018 -          ..
301019 -         Anytime, anywhere access to "intelligence" is accomplished by
301020 -         Communication Metrics...
301021 -
301022 -
301023 -                           http://www.welchco.com/
301025 -          ..
301026 -         The need for wide-scale testing is correct.  Educational
301027 -         testing was proposed for Communication Metrics to the
301028 -         University of Santa Clara on 951012. ref SDS 5 1940  On 970328
301029 -         U.S. Army Corps of Engineers reported Communication Metrics
301030 -         delivered useful "intelligence" that improved management.  On
301031 -         971007 the Corps reported cost benefit of Com Metrics was
301032 -         10:1, supporting prior limited testing by PG&E.  These
301033 -         successful limited tests support wider testing of Com Metrics,
301034 -         based on NSF award 9703470.
301036 -          ..
301037 -         NSF approved award 9616254 for a project that expired on
301038 -         October 31, 1999 for...
301039 -
301040 -              Efficient Algorithms for Learning and Reasoning from
301041 -              Data
301043 -          ..
301044 -         The abstract says this award is for...
301045 -
301046 -              ... development of ... algorithms that allow learning and
301047 -              limited reasoning over very large repositories of data.
301048 -              The goal is to combine these algorithms into an
301049 -              environment that supports summarization of very large
301050 -              streams of data (data mining) and the drawing of
301051 -              reasonably accurate conclusions from these summaries...
301052 -              (3) Incorporating knowledge in memory-based reasoning
301053 -              methods. The availability of high-speed, reliable,
301054 -              intelligent algorithms for learning and reasoning from
301055 -              data will have important applications in digital
301056 -              libraries, medical care, process control, and resource
301057 -              allocation, among other areas.
301058 -
301059 -
301061 -  ..
301062 - Appeal Procedures for NSF Grant Determinations
301063 -
301064 - Section 4.3 says...
301065 -
301066 -    Phase I proposals that have been declined or returned by NSF are
301067 -    not eligible for reconsideration under the same Program
301068 -    Solicitation, however, they can be resubmitted after suitable
301069 -    revision, under subsequent Solicitations. Phase II proposals that
301070 -    have been declined are not eligible for resubmission. ref OF 9 3068
301072 -  ..
301073 - This seems to indicate there is no procedure for NSF to correct
301074 - mistakes in the review process.
301075 -
301076 -     [On 991221 Sally suggests submitting the proposal next year.
301077 -     ref SDS 54 0001]
301078 -
301079 -
301080 -
301081 -
301082 -
301083 -
301084 -
301085 -
301086 -
301087 -
301088 -
301089 -
3011 -