'Evidence of the Afterlife' Supplemental Material Home Page

***
NDERF Home Page

***
Share Your Experience

The NDERF Study Methodology: A Closer Look
-Jeffrey Long, M.D.

Home Page Current NDEs Share NDE


NDERF Research Overview


            We will now take a closer look at how NDERF research is conducted.  This is important, as extensive NDERF research findings are presented throughout Evidence of the Afterlife: The Science of Near-Death Experiences.  NDERF research findings give a more in-depth look at many aspects of NDE than has been previously possible due to the vast number of NDEs studied.

            The NDERF website contains a survey with over 150 questions.  Responses from participants completing the survey are saved in the NDERF archives.  The survey data is placed in a database, and then merged into a word document.  The word document contains the survey questions and corresponding responses submitted from the website survey.  I personally review each submission to determine if the account meets is a near-death experience, as discussed in Evidence of the Afterlife: The Science of Near-Death Experiences.  Approximately half of all accounts submitted are determined to be NDEs.

            A large number of prior randomized studies have found that responses to Internet survey questions are as reliable as the traditional pencil-and-paper methodology.  This important point is discussed further in another section of this part of the website.
 

The NDERF survey contains a question asking permission to post the NDE on the NDERF website.  Permission to post their account on the website was given by virtually all.  Of those giving their permission to post their accounts, virtually all NDE accounts were actually posted.  NDE accounts may not be posted for ethical reasons if they strongly advocate illegal behavior, such as drug abuse or suicide.  Fortunately, this resulted in the exclusion of less than ten accounts from posting.  Approximately ' of 1% of all NDEs shared with NDERF were excluded from posting on NDERF for ethical reasons.  This percentage of excluded experiences is so small that this would not be expected to alter conclusions from the study of NDEs that are posted on NDERF.

            All but a very few accounts were posted anonymously.  The few accounts not posted anonymously were special situations where the NDEr gave explicit permission and there was a compelling reason to not post anonymously.  Examples of such situations include where the NDEr requested to share the title of a book they wrote, requested a link to another website with more information that identified them, or an emphatic request by the NDEr to not be posted anonymously.  Several NDErs gave approval for their account to be posted, then withdrew permission later.  In all such circumstances, the posted account was promptly removed.

            At the current time, many NDERF survey questions require check box responses of Yes/No/Uncertain.  Most questions in this format allow additional narrative responses.  Other questions allow only narrative responses.  Still other NDERF survey questions allow a range of responses.  All survey responses are placed in a database, allowing the responses to specific questions to be researched accurately even with a large volume of data. 

            Responses of NDErs to a number of NDERF survey questions have been researched in detail.  A summary of research conducted in this manner will be presented throughout Evidence of the Afterlife: The Science of Near-Death Experiences.

            In all prior NDE research, it has never been possible for interested individuals and researchers to review the original source of material; specifically the full NDE accounts.  In Evidence of the Afterlife: The Science of Near-Death Experiences, any interested reader may personally review the original source of material.  The response to each survey question may then be considered in the context of the entire shared NDE account.  In all prior published NDE studies, if a quote from a NDE account was given, readers could usually not review the context of the quote by reading the entire NDE account.  This is a very important advantage of the NDERF research methodology and helps in the understanding of the quotes presented in Evidence of the Afterlife: The Science of Near-Death Experiences

            This NDERF research methodology allows more accurate and unbiased understanding of the quotes presented from NDErs.  Most quotes presented in Evidence of the Afterlife: The Science of Near-Death Experiences were edited slightly.  NDErs were given a copy of the edited quote to review prior to publication.  The edited quote was only published in the book if the NDEr agreed that the edited quote was as accurate, or more accurate, than the unedited version of the quote that the NDEr's originally shared with NDERF.

            The NDERF research is based on studying the largest number of NDEs ever reported by using a relatively strict definition of NDE and using the NDE Scale.  Studying large numbers of NDEs allows more confidence in the study findings than would be possible from studying smaller numbers of NDEs. 

            No study of NDEs could possibly survey all NDErs everywhere.  Therefore, all research studies of NDErs are samples of a small portion of the total population of NDErs.  Consequently, there is some inherent statistical uncertainty in any survey of NDErs.  The large number of NDEs in the studies presented throughout Evidence of the Afterlife: The Science of Near-Death Experiences helps the confidence in the research findings of both quantitative and non-quantitative content of NDEs.

 

NDERF Research- An Even Closer Look

            More than 1300 NDErs have shared their experiences with the NDERF website over the years.  These NDErs completed the detailed NDERF website survey in addition to sharing a narrative of their experience.  This quantity of data provides the opportunity to make an important contribution to the scholarly understanding of NDE.  The largest previously published NDE studies generally involved analysis of less than several hundred NDEs.  From basic scientific principles, the NDERF study of large numbers of NDEs allows more confidence in many research findings than has been possible from previously published smaller NDE studies.

            Vast numbers of NDErs have taken the time to share the most important experiences of their lives with NDERF.  NDErs may spend over an hour completing the NDERF website survey.  Researching these NDEs using the best methods possible is an important way to respectfully honor the efforts of the NDErs who, out of the goodness of their hearts, shared with NDERF.  The NDErs never receive compensation of any kind for their efforts in sharing their NDEs with NDERF.

            Over the years, the cost of maintaining the NDERF website ranged between many hundreds to several thousand dollars per year.  Managing the day-to-day operation of the NDERF website requires hundreds of hours each year. In spite of this tremendous requirement of finances and time, I never personally had anything for sale on NDERF and never solicited contributions.  The NDERF website is dedicated to the accurate and comprehensive presentation of NDE.  Another major focus of the NDERF website has been to be a service to others. 

            An increasingly important role of NDERF is the scientific study of NDEs.  NDErs themselves are very interested in the scientific study of NDEs.  The NDERF survey asks 'What could a national organization with an interest in near death experience (NDE) do that would be of most interest to you?'  In response to this question, NDErs may select one or more of fourteen different responses.  The response most commonly selected by NDErs was 'Scientific study of NDE.'

            The following is an in-depth discussion of the methods used in NDERF research.  Understanding the NDERF research methodology will help explain why I am confident in the validity of the NDERF research results, and the remarkable conclusions of the NDERF study.

            Virtually all NDEs posted on NDERF are shared via the survey on the NDERF website.  A few of the posted NDEs are shared as narrative accounts via e-mail.  These accounts shared via e-mail did not involve completing the NDERF website survey.

            The NDERF website can be accessed world-wide, in every country of the world.  The NDERF website survey has been translated by volunteers into over 20 different non-English languages.  This allows the survey to be available in a familiar language to the great majority of people world-wide.  While the NDERF survey requires a computer, internet access, and the ability to find the NDERF website, this methodology allows the NDERF website survey to reach those who might not be easily reached by other survey methodologies.

            All experiences posted on NDERF were shared voluntarily by the NDErs.  Exceedingly few NDE accounts posted on NDERF are associated with any commercial interest of the NDEr, such as a book or for-profit website by the NDEr.  All commercial interests the NDEr may have that I am aware of are disclosed when the NDE is posted on NDERF.

            All prior surveys of NDErs required that the NDEr's be contacted in some manner.  Most prior surveys of NDErs had the limitation of surveying NDErs from a narrow geographic location and/or limited to members of a particular organization.  Prior surveys of NDErs are generally valid, but this illustrates the problems associated with trying to survey a representative sampling of all NDErs everywhere.

            The NDERF survey is accessible to people worldwide, and over a period of many years, with the only restriction being the need for a computer and internet access.  Thus, the NDERF survey methodology may be reasonably considered to allow as valid a sampling of NDErs everywhere as prior published surveys of NDErs.  Hundreds of other published NDE accounts are available (from sources other than NDERF) in books, scientific articles, newspaper stories, and magazine articles.  Substantial numbers of NDEs have been presented on television or radio.  Review of these NDE accounts indicates their similarity to NDEs posted on NDERF.  Many of the NDErs who shared with NDERF have also been on a variety of media presentations and/or have participated in other scientific studies of NDE.  This further supports the conclusion that the NDEs on NDERF are reasonably representative of all NDEs everywhere, and consistent with previously published NDEs.

            When NDEs are shared via the NDERF survey, responses are saved in a security protected part of the website.  The responses are downloaded from the website, usually weekly, and placed in a database.  The NDErs' responses are then merged into a document containing all the questions asked on the NDERF survey. 

            The NDErs' responses to the NDERF website survey are archived exactly as submitted.  For NDEs posted on NDERF, the accounts are edited only to correct obvious typographical errors, to remove specific information identifying the NDEr or other individuals by name, and to remove disparaging comments about specific individuals or institutions.

            NDErs are usually e-mailed a notice that their account is posted on the NDERF website, and a link to their account.  This is done both as a courtesy, and to give the NDEr a chance to review the accuracy of what is posted.  It is extremely rare that an NDEr requests any correction of their NDE as posted on NDERF.  This process helps assure that what NDErs share about their NDE is presented on the NDERF website as accurately and comprehensively as possible.  While this process takes a tremendous amount of work, it is vital to assure the reliability of the NDEs shared on NDERF.

            Two versions of the NDERF survey have been used for research.  The original version of the survey was used from January 1999 until September 2004.  This original survey was a very detailed questionnaire, allowing 131 possible responses to a variety of questions regarding NDEr demographics, NDE content, and NDE aftereffects.  A total of 413 NDErs meeting study criteria shared on this original version of the survey.

            For many questions, this original version of the survey allowed a response of 'Yes', 'Uncertain', and 'No', and a default response of 'No response'.  Many NDE researchers believe a significant percentage of NDEs are 'ineffable'; meaning not able to be expressed in words.  With the original NDERF survey, there was concern that NDErs might be unwilling to complete the survey if they did not believe a response of 'Yes', 'Uncertain', or 'No' was an appropriate answer for the survey questions. 

If NDErs were unwilling to complete the survey, the most important part of the survey, a narrative of their NDE, might never be available.  For the 413 NDEs analyzed from submissions to the original version of the NDERF survey, the option of 'No response' was selected between 2.9% and 15.2% for questions in this format, with an average of 7.0%.  After reviewing hundreds of NDEs shared on the original version of the NDERF survey, it became evident that NDErs were generally comfortable expressing their experiences in words.  This suggested the option of 'No response' was not necessary.

            Very few NDErs expressed concerned about the length of the original version of the survey.  Thus, the original survey was expanded.  Although some questions were eliminated, many new questions were added to help better understand NDE.

            In September 2004, the new version of the NDERF survey was launched.  The most important survey questions required a response from the NDEr.  This survey excluded the option of 'No response' for all questions.  The new survey allowed 150 possible responses to questions regarding NDEr demographics, NDE content, and NDE aftereffects. A total of 613 NDErs meeting study criteria, and with an NDE Scale score of seven or higher, shared on this new version of the survey.


            Another major change in the new version of the NDERF survey was the addition of 16 questions composing the 'NDE Scale'.  The NDE Scale is important and widely used in scholarly NDE research. This scale was developed in the early 1980's by Dr. Bruce Greyson.  The 16 questions in the NDE Scale ask about particular NDE elements, and their depth.  Reponses are scored from 0 to 32, with a score of 7 or above necessary to qualify as a NDE.  This allows NDE researchers to be more consistent in comparing the results of surveys of different groups of NDErs.

            The NDE Scale is not perfect.  This is illustrated in the following example of an individual crimially poisoned by strychnine (a highly toxic substance used in rat poison) and later told by paramedics she experienced cardiac and respiratory arrest:

It slowly dawned on me that something was terribly wrong. I watched the ducks swimming in the water... that was the last thing I saw, before I died' I went up, flew upwards to the crowns of the trees' I could see what was happening on the bridge, even though I was not on it. I saw that my boyfriend was not there yet. (Which he later confirmed).  Even though I was not on the bridge, I could see what was happening there. It was as if I could see anything I wanted to see. There were no limitations. I was still in the air'  (https://www.nderf.org/Experiences/1dominique_s_nde.html, NDERF Post database #2657)

This experience had a NDE Scale score of 6, which would not qualify as a NDE.  However, most people would accept this experience as a NDE. 

            Returning to the discussion of NDERF research methods, data from all NDEs shared on the NDERF survey are placed in a database and a spreadsheet.  Spreadsheet algorithms have been developed to allow automatic and rapid calculation of a variety of descriptive statistics as new data is added.  This is especially helpful as NDERF is accruing well over 200 new NDEs yearly, with the rate of accrual increasing.

           
Several strategies were prospectively developed to determine the validity of responses to the NDERF website survey.  Both the original and new versions of the NDERF survey were planned to contain redundant questions; questions asking the same concept in a slightly different manner.  The surveys were designed this way to help assess the consistency of NDErs responses to the survey questions.  For example, if NDErs completing the survey were not accurately responding to the questions, it would be expected to find poor internal consistency.  Poor internal consistency would be evident by significant variability in responses to similar questions.  On the other hand, if the respondents were accurately completing the NDERF survey, responses to questions should have good internal consistency.  For the responses to redundant questions reviewed, there was substantial consistency.
 
            My background as a physician helps me determine if the life-threatening event and subsequent course of medical events described in NDEs are medically plausible.  In the over 1300 NDEs shared with NDERF, a substantial number of NDEs described medical events that were surprising or even remarkable.  However, I have yet to review a NDE account shared with NDERF where the medical events described seemed so non-credible that I felt the account should be considered invalid.

            My medical background is important in confirming that experiences considered to be NDEs actually occurred in association with a medically life-threatening event.  NDE researchers lacking a medical background may encounter difficulty in making this critical assessment.

A major advantage of posting NDE accounts on NDERF is that this process essentially eliminates the possibility of posted NDEs being 'copycat' accounts.  A 'copycat' account means all or part of the NDE is copied, or plagiarized, from another source.  The NDERF website has well over 300,000 pages read by over 40,000 unique visitors to the website each month.  If any posted experience is a 'copycat' account, it is highly probable it will be identified by one or more of the vast numbers of NDERF website visitors.

In the history of NDERF, only one copycat account was ever posted.  This occurred in the first year of NDERF's existence.  This account was shared via a personal interview, a process we no longer use, and was not shared via the NDERF survey.    After this account was posted, we were promptly notified by a website reader that this account duplicated significant portions of a previously published NDE.  The copycat account was immediately removed.  If, at any future time, any NDE posted on NDERF is found to be a copycat account, it will be excluded from NDERF research.

            The enormous number of people reading NDEs posted on NDERF helps assure their validity.  This is a significant advantage unique to the website survey research methodology conducted by NDERF.  The NDERF website has an associated public bulletin board, called the 'Forum'.  The Forum began several years after the NDERF website was started.  The Forum is quite active, with well over 100,000 pages read monthly.  This provides another mechanism to present any concerns about the validity of any posted account.

 
            The NDERF survey itself is a strong disincentive to share falsified accounts.  There were 131 possible responses on the original NDERF survey, and 150 possible responses in the new NDERF survey.  The new survey requires responses to all key questions; thus it is not possible to skip many survey questions.  It probably takes at least 45 minutes, possibly much longer, to read and respond to the survey questions and type in narrative responses.  A confabulated account would almost certainly not be as clear in the mind of the contributor as the remembrance of a real NDE.  It would be enormously difficult to consistently respond to the detailed survey questions based on a confabulated account. 

Only two clearly 'joke' accounts have ever been submitted to NDERF.  These accounts were submitted in the name of a well known adult movie actress, submitted at approximately the same time, and contained responses to survey questions that were clearly intended to be a 'joke.'  This is a startlingly low percentage considering over 1300 bona fide NDEs have been shared with the NDERF survey over the years.  The NDERF survey appears so complex and time consuming that 'joke' submissions are overwhelmingly discouraged.

NDEs are generally posted on NDERF anonymously as an ethical necessity.  Posting NDEs anonymously also serves as a substantial disincentive to share a fraudulent account, as the contributor would never receive personal recognition for their deceit.  We have yet to post an account where we were later contacted by the contributor to inform us the account was fraudulent.

            Some NDErs who have never shared their experience with another person completed the NDERF survey.  A NDERF survey question asks 'Have you shared this experience with others?'  Of 613 NDErs,  90.4% responded 'Yes',1.1% responded 'Uncertain', and 8.5% responded 'No.' The 8.5% responding 'No' to this survey question is a small, but non-trivial percentage of NDErs.  Reaching this group of NDErs, who might otherwise be unwilling to share their NDE and participate in NDE research, represents another advantage of the NDERF website survey methodology.

Respondents to the NDERF survey must contribute the substantial effort of typing out their own responses to the survey questions.  Thus, NDErs completing the NDERF survey may be more motivated to share their experience than NDErs surveyed via personal interviews, where less personal effort by the NDErs is required.

NDEs are complex experiences that might be difficult to express in words.  The NDERF survey requires NDEs to be shared in written form with words.  Fortunately, the NDErs completing the NDERF survey are generally confident with their ability to accurately convey their NDE.  A NDERF survey question asks 'Did the questions asked and information you provided accurately and comprehensively describe your experience?'  This question is intentionally worded to encourage a response other than 'Yes' if there is any aspect of their NDE that the NDEr believes is not accurately conveyed on the survey. 

In the original version of the NDERF survey, 378 NDErs responded to the survey question.  Of these, 312 (82.5%) responded 'Yes', 38 (10.1%) responded 'Uncertain', and only 28 (7.4%) responded 'No.'  In the new version of the survey, response to the preceding question was required.  Of 613 NDErs responding to the question, 518 (84.5%) responded 'Yes', 54 (8.8%) responded 'Uncertain', and only 41 (6.7%) responded 'No.'  Responses to this question strongly suggest NDErs themselves are generally satisfied that the NDERF survey, and the answers they give, accurately and comprehensively conveys their experience. 

            This is by far the largest group of scientifically studied NDEs with the use of the NDE Scale ever reported.  As noted previously, larger numbers of NDErs surveyed allow greater confidence in the survey results.  This is applicable to studying the non-quantitative narrative content of the NDEs.  The large group of NDEs in the NDERF study allows more confidence in the findings of both quantitative and non-quantitative NDE content than has been possible in any previous study.

            It is unlikely that any other NDE research studies published in the foreseeable future will be able to include as many NDEs as the NDERF study.  It is enormously difficult to access large numbers of NDErs who are willing to participate in a study.  The NDERF website, due to its enormous visibility on the internet and excellent reputation, is uniquely positioned to encourage NDE contributions for scientific study.  There are approximately 48,000
links on the internet to the NDERF website.  Even with NDERF's exceptional visibility on the Internet, it took over 9 years to receive 1300 NDEs.

            Previously published NDE research studies often relied on excerpted quotes of NDErs as a major line of evidence for their conclusions.  Publishing excerpts from the NDE accounts was usually necessary because the full text of the NDEs were usually too long to publish due to space constraints.  In these previously published NDE studies, there was generally no possibility for others to review the context of quoted NDEs by reading the entire NDE.  In the NDERF study, readers have full access to the original source of data.  This should significantly increase confidence in the findings of the NDERF study.  This is especially important as many of the findings of this study are profound.