braintv: a novel approach for online mapping of human brain functions · 2009-08-03 · frequency...

401LACHAUX ET AL. Biol Res 40, 2007, 401-413Biol Res 40: 401-413, 2007 BRBrainTV: a novel approach for online mapping of humanbrain functions

JEAN-PHILIPPE LACHAUX, KARIM JERBI1, OLIVIER BERTRAND1,LORELLA MINOTTI2, DOMINIQUE HOFFMANN3, BENJAMIN SCHOENDORFF1,and PHILIPPE KAHANE2

1 INSERM, U821, Lyon, F-69500, France; Institut Fédératif des Neurosciences, Lyon, F-69000, France;Université Lyon 1, Lyon, F-69000, France.2 Department of Neurology and INSERM U704, Grenoble Hospital, Grenoble, France.3 Department of Neurosurgery and INSERM U318, Grenoble Hospital, Grenoble, France.

ABSTRACT

Our understanding of the brain’s functional organisation has greatly benefited from occasional exploratorysessions during electrophysiological studies, trying various manipulations of an animal’s environment totrigger responses in particular neurons. Famous examples of such exploration have unveiled variousunexpected response properties, such as those of mirror neurons. This approach, which relies on thepossibility to test online the reactivity of precise neural populations has no equivalent so far in humans. Thepresent study proposes and applies a radically novel framework for mapping human brain functions inecological situations based on a combination of a) exploratory sessions, using real-time electrophysiology toformulate hypotheses about the functional role of precise cortical regions and b) controlled experimentalprotocols specifically adapted to test these hypotheses. Using this two-stage approach with an epileptic patientcandidate for surgery and implanted with intracerebral electrodes, we were able to precisely map high-levelauditory functions in the patients’ superior temporal lobe. We propose that this procedure constitutes at theleast a useful complement of electrical cortical stimulations to map eloquent brain areas in epileptic patientsbefore their surgery, but also a path of discovery for human functional brain mapping.

Key terms: speech, gamma rhythm, beta rhythm, brain computer interface, neurofeedback, neuralsynchronization.

INTRODUCTION

For patients with intractable epilepsy whorequire brain surgery, the success andsafety of the procedure crucially dependson the correct definition of the brainvolume to be resected. This requires notonly the identification of the epileptogenicnetwork but also a precise understanding ofthe functional organisation of the cortex inwhich it is embedded, so that eloquentcortical regions can be spared (Lesser et al.,1998). Due to large interindividualvariations in functional corticalorganisation (Ojemann et al. , 1989),especially when lesions are present, precise

mapping between cortical structures andfunctions must be performed for eachindividual patient, and cannot be based onthe sole functional neuroimaging literature.

To date, most clinical epilepsy centresrely on electric cortical stimulations (ECS)to reveal the functions of implanted corticalareas, due to the direct effect electriccortical stimulations have on the underlyingprocesses carried by the neural populationstimulated (Lesser et al., 1998). Thisprocedure has proved very effective inmapping several functional systems, suchas the motor system, the vestibular systemor several aspects of the language system,among others (Kahane et al., 2003, Lesser

Correspondence should be addressed to: Jean-Philippe Lachaux ([email protected]) Mental Processes and BrainActivation INSERM - Unité 280, Centre Hospitalier Le Vinatier, Bâtiment 452, 95 Boulevard Pinel, 69500 BRON,FRANCE, Phone : 33 4 72 13 89 00, Fax : 33 4 72 13 89 01

Received: May 11, 2007. Accepted: March 5, 2008

LACHAUX ET AL. Biol Res 40, 2007, 401-413402

et al., 1998, Lobel et al., 2001, Luders etal., 1995, Ojemann et al., 1989, Penfieldand Jasper, 1954). However it may not bebest suited to exploring complex cognitivefunctions which are unlikely to be active atthe time of stimulation, such as mentalcalculation for instance.

Another approach is to engage patientsin cognitive protocols involving thecorresponding systems and test forassociated modulations in the signal of theimplanted electrodes (Lachaux et al., 2003).In particular, it has been suggested recentlythat task-induced energy modulations ofintracranial EEG in the high gamma band(> 70 Hz) constitute an efficient marker forcognitive mapping that can usefullycomplement the standard ECS approach(Sinai et al., 2005). Indeed, over the lastfew years, it has become increasingly clearthat most cognitive processes elicit, withintheir underlying cortical networks, highlyanatomically and functionally selectivegamma band energy modulations. To citeonly human intracranial studies, this hasbeen demonstrated for memory (Fell et al.,2001, Howard et al., 2003, Mainy et al.,2007), visual attention and perception,audition (Bidet-Caulet et al., 2003, Crone etal. , 2001, Edwards et al. , 2005),somatosensory and motor processes(Lachaux et al., 2005, Pfurtscheller et al.,2003, Szurhaj et al., 2005) and language(Crone et al. , 2001). These studiesexpanded on a large field of research ongamma band activity in animals (Singer,1999) and in human scalp EEG studies(Tallon-Baudry and Bertrand, 1999).

We have used this Dynamic SpectralImaging (DSI) approach (i.e. the timefrequency analysis of intracerebral EEGactivity to extract task-induced gamma bandmodulations) for the past few years, incomplement with ECS, and our conclusion isthat DSI is very efficient to identify whichparticular cortical regions are involved in apre-defined cognitive process, but lessefficient to answer the reverse question, thatis, which cognitive process is mediated by apre-defined cortical region. This comes fromtwo limitations of DSI, as it is presently used.

The first limitation comes from theexquisite functional specificity of the

gamma band, which leads to the fact that, inour experience, a given cognitive task, suchas encoding and maintaining a list of lettersinto working memory, triggers gammaenergy modulations in less than 10% of allthe recorded sites (Mainy et al., 2007).Since the anatomical distribution of thosemodulations varies with the task, it istheoretically possible to test the patient in alarge repertoire of tasks to eventuallyactivate most of the sites and find out theirfunctional specificity. However, this is along procedure that may proveuncomfortable for the patient.

The second limitation is that thisapproach can only reveal cognitiveprocesses for which protocols have beentailor-made in advance. The design of acognitive task is a complex process thesuccess of which depends on a carefulchoice of control conditions. The number ofcognitive processes that may be carried bythe recorded sites vastly exceeds thenumber of protocols that can reasonably bethought of and set up by a single clinicalgroup. Furthermore, there is no way toanticipate all the possible situations that cantrigger activity in a particular brain area. Asa matter of fact, many important discoveriesin electrophysiology came through openexploration, as happened with the discoveryof mirror neurons in the frontal lobe ofmacaque monkeys (di Pellegrino et al.,1992). The use of a set of standardprotocols is thus not necessarily best suitedfor exploration.

To a certain extent, the usefulness ofECS mapping comes from the fact that it islargely exploratory: it relies on the verbalreports of the patient describing the effectsproduced by the stimulation, or theobservation of electrically-elicitedbehaviours. Those reports can potentiallyreveal cognitive processes that had not beenenvisioned beforehand. The objective ofthis study was to develop an approachbased on DSI that would allow for such anexploratory phase. Increments in computingpower have fuelled resurgent interest in theapplication of online analysis of humanbrain activity. To date, most of the researchhas focused on one particular application,the transformation of neural measures into

403LACHAUX ET AL. Biol Res 40, 2007, 401-413

commands to control an external device(Brain Computer Interface) (Vaughan et al.,2003). However, the range of applicationsof these techniques is even wider. Inparticular, they can provide a way toexplore the behaviour of neural measures inreal-time, while a subject engages intovarious activities.

The present study proposes andillustrates a functional mapping procedurecentred on neurofeedback. We designed asimple computer interface to display inreal-time to the patients their activityrecorded at particular cortical locations inseveral frequency bands including the alpha(8-12 Hz), beta (12 - 30 Hz) and gammabands (> 40 Hz). During such neurofeedbacksessions, this device allowed us to identify inan epileptic patient a selectivity for speechin the mid portion of his superior temporalcortex, for sentences or words, but not forshort nonsensical speech patterns such as‘patapatapata’, or other vocal sounds or thepatient’s own speech. Those observationswere statistically validated in subsequentsessions using experimental paradigmstesting specifically those responseproperties.

METHODS

Patient and recordings

The patient suffered from drug-resistantpartial epilepsy and was a candidate forsurgery. Magnetic resonance imaging of thebrain (MRI) showed a left hippocampalsclerosis. Because the location of theepileptic focus could not be identified usingnon-invasive methods, the patientunderwent intracerebral EEG recordings bymeans of stereotactically implantedmultilead depth electrodes (SEEG) (forexplanation of this methodology, see(Kahane et al., 2004), on the basis of whichthe epileptogenic zone proved to be leftantero-mesial temporal. Selection of sites toimplant was solely based on clinicalconsiderations with no reference to thepresent experimental protocol. The patientperformed the experimental task 11 daysafter implantation of the electrodes, and had

previously given his informed consent toparticipate in the experiment.

Twelve semi-rigid electrodes wereimplanted in cortical areas related to thesuspected origin of the seizures. Eachelectrode had a diameter of 0.8mm (shownon figure 1) and comprised 5, 10, 15 or 182mms long leads, 1.5mm apart (Alcys,Besançon, France), depending on the targetregion. The electrode contacts wereidentified on the patient’s individualstereotactic scheme, and then anatomicallylocalized using Talairach and Tournoux’sproportional atlas (Talairach and Tournoux,1988). In addition, computer-assistedmatching of post-implantation CT-scan witha pre-implantation 3-D MRI afforded adirect visualization of the electrode contactswith respect to the patient’s brain anatomy(Voxim, Germany). Intracerebral recordingswere obtained using an audio-video-EEGmonitoring system (Micromed, Treviso,Italy), allowing for the simultaneousrecording of 128 depth-EEG channelssampled at 512 Hz [0.1–200 Hz bandwidth].

Intracerebral electrical stimulations wereperformed for purely clinical purposes overa few days, during sessions that lasted 1 to2 hours and under video-SEEG control.Following our standard clinical practice(Kahane et al., 1993), stimulations at 1Hz(3 mA) and 50 Hz (1-3 mA) were appliedbetween contiguous contacts at differentlevels along the axis of all the electrodes.Bipolar stimuli were delivered using aconstant current rectangular pulse generatordesigned for a safe diagnostic stimulationof the human brain (Micromed, Treviso,Italy), according to parameters proven toproduce no structural damage.

PARADIGM

Neurofeedback session

The neurofeedback consisted of presentingthe patient with his imaged cerebral activityin four cortical sites at a time, appearing ascurves displaying energy levels measured inseveral frequency bands over the previousten seconds (Figure 1). This simple deviceallowed the patient easily to focus on one


Figure 1: BrainTV setup. a) depth electrode used for intracranial EEG recordings in epilepticpatients, it comprises 15 recording sites evenly spaced every 3.5 mm; depth electrodes areorthogonal to the sagittal plane : typical entry points are shown as black dots on the 3D brainreconstruction (b). c) difference between the classical approach used in standard human brainmapping studies, and the brainTV approach. In the former, the goal is to find all brain regionsassociated with a predefined cognitive process; in the latter, the goal is to find all cognitiveprocesses associated with a predefined brain region. d) BrainTV display : the patient is shown witha computer screen displaying, for two frequency bands ([8-30 Hz] in blue and [60-140 Hz] inorange) the energy measured in four cortical sites over the last 10 seconds (the four panels, withcorresponding site names indicated in white). Displays are continuously updated and always showsignals recorded in the last ten seconds. The energy is expressed in units of the standard deviationof the energy measured at this frequency in a reference baseline period. The display is on during theentire day and the patient decides when to watch and interact with it. Using a remote control, thepatient can decide which part of his cortex is being visualized, literally surfing between recordedchannels; a yellow dot on a simplified representation of the brain informs the patient of theanatomical origin of the visualized activity (upper part of the display).


particular neural marker. Frequency bandsdisplayed could be adjusted to match theelectrophysiological signature of therecorded site. In this experiment, twofrequency bands were chosen ([8-30 Hz]and [60-140 Hz]) to cover the alpha, betaand gamma bands while avoidingcontamination by 50 Hz noise. Note thatsince alpha and beta bandwidths had beenfound to be extremely correlated inprevious studies by our group, they were(additively) pooled together in one singlecurve. The software also allowed theexperimenter to set activity display with alag of 30 seconds. This allows for testingthe validity of a patient’s report that s/hecould control his/her brain activity, as theysometimes misleadingly believe. Theneurofeedback period consists of threesessions : a) a ‘free search’ session duringwhich the patient is left alone with thefunctioning display which it can at his/herown leisure, observe and possibly noticesystematic correlations with exogenous orendogenous events, b) a ‘guided search’session, which, in case of failure of the freesearch, involves orienting the patient’sattention to cognitive processes putativelyeffected by the cortical location underinvestigation. The ‘guided search’ sessionis based on the existing neuroimagingliterature and on the results of previouscognitive experiments that the patient mayhave participated in. c) a ‘confirmation’session, which follows the previous ones incase of success. The aim of thisconfirmation session is to test and refineany correlation found with the patient.Crucially, the success of the wholeprocedure depends on the patient’ssustained motivation and interest.

Experimental protocol

Optimally, the confirmation session mustbe followed by a classic experimentalsession during which the patient performs acognitive task centred on the cognitiveprocesses identified through neurofeedback.This task can be either drawn from a (local)large bank of protocols (to date, forinstance, twenty paradigms are ready-to-usein our center), or especially designed in the

time remaining before the removal of theelectrodes. Thus, neurofeedback sessionsshould ideally be conducted in the earlydays of the patient’s implantation. Thepresent experiment was designed to contrastsix experimental conditions : a) patientlistening to sentences spoken by theexperimenter in his mother tongue(READ_EXP), b) patient reading aloudequivalent sentences in his mother tongue(READ_PAT), c) patient l istening tosentences spoken by the experimenter inFinnish (Suomi), a language unknown bothto him and the experimenter, (SUOM_EXP),d) patient listening to repeated nonsensicalshort speech patterns pronounced by theexperimenter (‘patapatapata…’ or‘mumomumomumo….’) (PATA_EXP), e)patient speaking self-generated sentencesdescribing recently past events(TALK_PAT) and f) patient listening todrum patterns produced by theexperimenter (NOIS_EXP). The protocolwas block-designed, with 10 to 20 stimulusperiods of about 10 seconds for eachcondition. Stimulus periods were separatedby at least one second and a time-markerindicated the end of each stimulus periodand was used for averaging.

DATA ANALYSIS

Neurofeedback data analysis

The neurofeedback program displayed thespectral energy of the signal of oneparticular bipolar derivation computedbetween adjacent electrode contacts. Onecycle of analysis procedure consisted of a)applying a short-term Fourier transform tothe signal (using a sliding Hammingwindow of 256 milliseconds moving bysteps of 128 ms) to extract the time-courseof its spectral energy over the last 10seconds for all frequencies between 1 and150 Hz, b) normalizing this time-frequencyenergy map (z-score) independently foreach frequency, relative to the spectralenergy measured during an earlier baselineperiod (while the patient was at rest,motionless, with his eyes opened in a quietsurrounding), c) averaging the TF energy in


three frequency bands ([8-30 Hz], [60-90Hz], [110-140 Hz]), d) adding the [60-90Hz] and [110 – 140 Hz] components (theaim is to cover the [60 -140 Hz] frequencyband while avoiding 50 Hz harmonics at100 Hz, e) applying convolution in timewith a two seconds time window, toincrease the signal over noise ratio of thedisplayed signal and finally f) displayingthose two 10-second energy time courses inthe form of two plots (figure 1; gammashown in orange, alpha/beta in blue). Thesoftware was set to refresh the display asoften as possible, that is, to get newlyacquired data and analyze them as soon asthe previous cycle of analysis-display hadbeen completed. The program ran underMatlab (The Mathworks, inc) on a PCaccessing the acquisition PC disk via anEthernet connection. In effect, the refreshrate of the display is limited solely by theacquisition software (Micromed, Treviso,Italy) which writes new data on theacquisition hard drive every 250 ms.

Experimental protocol analysis

For each single trial showing no sign ofepileptiform activity, bipolar derivationscomputed between adjacent electrodecontacts were analyzed in the time-frequency (TF) domain with a MorletWavelet transform, following our usualprocedure (Mainy et al., 2007) as in theneurofeedback session. Investigatedfrequency range was [1-200 Hz] and thetime range between 3000 ms before and2000 ms after the time marker (set as theend of the stimulus period).

The analysis was designed to be as closeas possible to that used during theneurofeedback session. For each trial, theTF energy map was normalized (z-score)independently for each frequency relative tothe mean and standard deviation of acommon baseline. More precisely, thisnormalization consisted in a) computing themean and standard deviation of all theenergy values measured during the [200 ms:400 ms] baseline in all the trials (for all theconditions), b) subtracting this mean energyfrom the TF map under process, and c)dividing by the standard deviation

calculated above. Because the mean and thestandard deviation were computed frombaseline energy values pooled across all sixexperimental conditions, normalization wassimilar for all six maps. Given that duringthe neurofeedback session, the effects hadbeen mostly observed in the [60-90 Hz] and[110 - 140 Hz] ranges and with a 2000 msaveraging window, the statistical analysiscompared across the six conditions themean TF energy in the ([60-90 Hz] and[110-140 Hz]) x [-2000 ms : 0 ms]) TF tileobtained in all the individual trials (i.e. themean of energy values measured in both the[60-90 Hz] and the [110-140 Hz] ranges).

Statistical analysis of TF energy valueswas performed with a nonparametricKruskal-Wallis test and Fisher’s protectedleast significant differences based on ranks(FPLSD) for post hoc tests of significance(Portney and Watkins, 1993). The use ofnonparametric tests was necessary becauseof the far from Gaussian distribution of TFenergy values. It was performed on rankeddata and provided a H value indicatingwhether a global effect of stimulation typewas significant. If found significant, FPLSDtests allowed for comparisons of all thepossible combinations of experimentalcondition pairs, and thus to determine inwhich pairs significant differences occurred.

For visualization purposes (Figure 2),the TF maps were averaged across all trialsin a given experimental condition andnormalized through the procedure describedabove.

RESULTS

Neurofeedback session

The neurofeedback consisted of presentingthe patient with a display of his own brainactivity (figure 1). In this case, imagedactivity was presented for site A (notvisible in the display in figure 1) in the midpart of the upper bank of the left superiortemporal sulcus (mSTS) (Talairachcoordinates: -65, -10, 5 mm) (figure 2),together with three other sites. The displayappeared as two dynamic curves displayingenergy levels measured in two frequency


Figure 2: Spectral Energy variations associated with speech in the mid superior temporal sulcus(site A in this study). Each map represents for each frequency below 200 Hz, the time course ofspectral energy measured in site A (Talairach coordinates : -65, -10, 5) over the last two seconds ofauditory stimulation (black horizontal bar) and over the first second after. The six graphscorrespond to the six experimental conditions. At each frequency the energy is expressed in units ofthe standard deviation of the energy measured at this frequency in the [200 ms : 400 ms] period.The normalization parameters were the same for all maps. Recording site is represented on a 3Dreconstruction of the Mni single-subject’s brain template.

bands ([8-30 Hz], [60-140 Hz] over theprevious ten seconds.

The Neurofeedback session started witha free search period, during which wesimply let the patient become familiar withvariations of the display. The patient was

informed that the display represented hiscerebral activity in real time, but noinformation was given pertaining to theputative function of the cortical area. Weencouraged the patient to be alert topossible temporal correlations between the


marker and events in his environment or inhis own experience, both at rest and duringusual activities such as reading. The displayrepresenting the last 10 seconds of corticalactivity, made it possible for the patient toswitch his attention from time to timebetween the task at hand and the display tonotice correlations. Site A was chosenbecause it had earlier evidenced, for thispatient, a gamma-band energy increase tovisually presented words and pseudo-wordsduring a silent reading task. We thereforeanticipated that, while reading his book, thepatient might notice an increase of themarker displayed .

This was not evident to the patient. So,after 10 minutes of free search period,conducted in silence, the experimenterentered the room and asked the patient if hehad noticed anything, eliciting a negativeanswer, including for reading. However, theconversation itself triggered a strongincrease of the index, first interpreted as ageneral arousal effect, but which quicklyturned out to be more specific. Threeminutes of exchange between experimenterand patient followed, centred on therelationship between the marker andreading, during which both persons readaloud in turn. After this period, it was clearthat the marker reacted selectively to theexperimenter’s speech The neurofeedbacksession went on for another thirty minutes,during which multiple members of the staffwent in to try out multiple tests, such asrepeating nonsense speech patterns(‘patapatapata’ : no response of themarker), or talking in multiple languages(spoken or not by the patient : response,sometimes stronger than for French, such asfor German, which the patient understood atschool level). At the end of this session, itwas evident to the staff that the region wasmost strongly activated when the patientwas attentive to spoken sentences, whetherin known or unknown languages. Incomparison, the region was not activatedeven by louder noises, such as drumpatterns, or by the patient’s own speech,predictable speech patterns such as thealphabet, or by loud conversations betweenthird parties while the patient was talking tothe experimenter.

Experimental session

The design of the experiment was adaptedto the observations made during theneurofeedback session, and had the patientlisten to sentences pronounced in hismother tongue (READ_EXP), in anunknown language (SUOM_EXP), or tonon-sensical speech patterns (PATA_EXP)or loud noises (NOIS_EXP) or readsentences in French (READ_PAT) orsimply talk (TALK_PAT). Figure 2 showsthe time-frequency energy maps for thesesix conditions, normalized relative to acommon baseline. The maps, whichrepresent the energy averaged over themultiple trials of each condition, show aclear decrease in gamma band activity atsentence end in two conditions only(READ_EXP and SUOM_EXP).

To quantify this effect, we comparedbetween conditions the energy recorded atsite A in the [60 - 90 Hz] and [110 - 140Hz] ranges averaged over the last twoseconds of each speech or noise period.This choice was motivated by the fact thatthe most efficient marker during theneurofeedback sess ion had been theenergy of the past two seconds in thoseranges. Figure 2 shows the gamma bandact ivat ion e l ic i ted dur ing the s ixconditions. A Kruskal-Wallis analysisrevealed a s igni f icant ef fec t of theexper imenta l condi t ion (H = 45.7 ,p<0.0001). Post-hoc Fisher’s protectedleast significant differences test appliedon ranks showed that the strongest energywas obta ined in the SUOM_EXPcondition (SUOM_EXP > READ_PAT,PATA_EXP, TALK_PAT andNOIS_EXP, p<0.05) , the d i f ferencebetween the SUOM_EXP and theREAD_EXP was not s igni f icant ( inaddition, we observed READ_EXP >NOIS_EXP, p<0.05). For information,Mann-Whitney comparisons between theREAD_EXP and the other conditionsindicated a d i f ference wi th theREAD_PAT, PATA_EXP and NOIS_EXP(p<0.01). The same test was significantbetween the READ_EXP and theSUOM_EXP conditions (p<0.05), withSUOM_EXP > READ_EXP.


Effects of Electric Cortical Stimulations

A first session of ECS took place beforethe neurofeedback session and revealed nonoticeable effects on site A, either at 1 Hzor at 50 Hz (up to 3mA) (the patient had tocount aloud, or listen to the experimentercounting, ‘did you hear me correctly ?’).However, the behaviour of this site duringthe neurofeedback led us to stimulate thissite in a second session during which wetested specifically the patient for speechperception impairment, and this time, it wasnoticed that 50 Hz stimulations (3mA)produced difficulties in sentenceunderstanding.

DISCUSSION

The main outcome of this study is todemonstrate that gamma band energymodulations could be used as onlinemarkers of high-order cognitive processesin humans. To our knowledge, thisobservation has thus far only been reportedfor lower frequency activities or in themotor cortex and early sensory regionsduring simple movements or sensorystimulations (Chatrian et al. , 1960,Graimann et al., 2004, Leuthardt et al.,2004, Pfurtscheller et al., 2006), that is, inregions whose functional organisation isalready well-known and which can bereadily identified using ECS. Our resultsdemonstrate the validity of neurofeedbackfor the identification of certain eloquentbrain areas in humans, in conjunction withstandard ECS. Both techniques should bethought of as complementary: theneurofeedback led us to specify precisefunctions whose perturbation had to betested during the ECS session, and ECSrevealed the effects on those functions oflocal perturbations of neural activity. Theidea is not to replace ECS byneurofeedback sessions, but to complementit for certain functions.

It should also be noted that, as inprevious intracranial studies, it was in thegamma band that the correlation with thepatient’s behaviour proved to be tightest.Although the visual display also presented

the activity in the alpha and beta bands, thepatient spontaneously used the gammaactivity as the more efficient marker. Thecortical origin of those gamma bandmodulations was consistent with the well-known implication of the mid superiortemporal sulcus in speech perception.Several neuroimaging studies have clearlyestablished that this region activates insituations contrasting vocal vs. non vocalsounds (Belin et al., 2002, Belin et al.,2000), or human vs. animal vocal sounds(Fecteau et al., 2004). Indeed, in an fMRIstudy comparing short vocal vs. non-vocalsounds (Belin et al., 2002), the maximumBOLD difference was found only a fewmillimetres away from recording site A inthe Talairach space (Talairach: -62, -14, 0).Also, a recent study (Cohen et al., 2004)has proposed that a region very close to ourrecording site (Talairach: -60, -8, -4) mayconstitute an auditory analog of the VisualWord Form Area. The effect we observed isthus fully compatible with theneuroimaging literature and not surprising aposteriori, however, we believe that theexact behaviour of site A during theneurofeedback session could not have beenknown a priori, not only because of theinterindividual variability, but also becausethe exact function of this region in speechperception is still a matter of debate (Scottand Johnsrude, 2003). Several studies haveproposed that i t may be involved insentence processing (Friederici et al., 2003,Humphries et al., 2005), either for thesyntactic or for the semantic analysis ofsentences (Friederici et al., 2003). In thepatient we examined, this region respondedstrongly and selectively to sentencespronounced by the experimenter (in Frenchor in an unknown foreign language), andless by the patient, and did not respond tonon-sentence-like speech patterns such asthe alphabet. This pattern of responses isvery consistent with a participation in high-level phonological decoding, which wouldexplain why there was no response to thesubject’s own voice, because the patient didnot decode his own sentences. However, itis unlikely that the region participated insemantic processing because the strongestresponse was obtained for sentences in


Suomi, a language unknown to the patient.Interestingly, the activity in this region hasbeen found to be altered in autistic children(Boddaert et al., 2004), which may explaincertain difficulties in communicating withothers. In our sense, those results show thatthe neurofeedback session actually broughtnew insights into the function of the midSTS during speech perception, and not asimple confirmation of the knownfunctional anatomy of this region.

Altogether, our results support the use ofneurofeedback for exploratory purposes,during functional cortical mapping. Everybrain mapping study is based on theexploration of a ‘stimulus-task’ space,achieved through manipulations of both theenvironment of the subject and the task s/heis instructed to perform. Cognitive protocolsrely on a parameterization of a subregion ofthat space, which then allows for correlatingvariations of brain activity measurementswith variations of those parameters. Becausecognitive experiments have to be relativelyshort, there is always a compromise betweenthe size of the subregion explored and thedegree of precision with which it is explored(a study on face perception can test thecerebral sensitivity to subtle differences inface shape, but does not reveal anythingabout the response to landscapes).

The approach that we propose is anattempt to overcome this constraint via amultiscale and stepwise exploration of the‘stimulus-task’ space. The initialneurofeedback session allows for coarseexploration of very large portions of thatspace. Exploration then narrows down tomuch more specific and ‘efficient’experimental manipulations eventuallyyielding a precisely controlled cognitiveprotocol. In fact, this approach is parallel tothe mapping of receptive fields in V1, andcan be thought of as a ‘cognitive fieldmapping’ procedure. Note that the initialneurofeedback exploration may begenerally sufficient to convince bothexperimenter and patient of the functionalrole of the investigated area. Therefore, theuse of precisely controlled protocols andprecise statistical evaluations may not bemandatory in standard clinical practice, butonly necessary for research purposes.

An important methodological question iswhether the initial exploration should beguided or whether the patient should be leftalone with the feedback display on, to trywithout help to find correlates of his/herbehaviour and experience. The answerlargely depends on the time available forexploration, and on the patient ability forintrospection. For a quicker search, it maybe helpful to encourage the patientspecifically to explore certain cognitiveprocesses in particular, based on the generalknowledge gained from functionalneuroimaging of the cortical area underinvestigation. In this regard, one shouldnote that recent studies have shown a clearcorrelation between the recordings ofBOLD signal and gamma activity (Kim etal., 2004, Lachaux et al., in press, Mukamelet al., 2005, Niessing et al., 2005), whichsuggest that the fMRI literature can be usedto narrow down the patient’s exploration tospecific cognitive processes. However, thefree search phase potentially allows to testfor effects not shown yet with fMRI, andalso to deal with possible corticalreorganisations specific to an individual.

A further question is whether thisapproach can be used for all structures andfunctions. It is well known to workefficiently for simple sensory and motorprocesses, and our results demonstrate itsvalidity for high-order auditory processes,but it is not clear yet whether it is adaptedfor more endogenous mental events, such asslight emotional or attentional modulations.The limits of the approach are in factdetermined by two factors a) theintrospective capacity of the patient toidentify in his/her own subjectiveexperience events correlated in time withthe feedback neural marker and b) theintrinsic variability of the neural marker.

The first point is at the center of a largerresearch project known as‘neurophenomenology’ (Petitot et al., 1999,Thompson and Varela, 2001, Varela andShear, 1999, Varela, 1996) which exploresthe advantages and limits of combining‘first-person data’ (subjective reports) with‘third-person’ neuroimaging data. It isenough here to say that this project posesmethodological challenges, which have


been addressed by several authors (Varelaand Shear, 1999). We will simply note herethat the procedure can greatly benefit fromthe assistance of an expert in guiding thepatient’s introspection (Petitmengin, 2006).At the very least, it is fair to say that theprecision of information the patient canprovide is directly influenced by the time s/he has had to familiarize herself/himselfwith her/his displayed neural activity. Notethat the different site markers need not tobe shown in turn, one by one, as in thepresent experiment (although this allowsboth the patient and the experimenter tofocus on one single site), but one mayenvision to display continuously theactivity of several sites simultaneously(visually, or through sonification) and letthe patient become familiar with them atleisure. Another point raised by theneurophenomenological approach iswhether the experimenter can validly trustpatients reports (Jack and Roepstorff,2004), which is a real issue when the neuralmarker correlates with mental eventsaccessible only to the patient. At somepoint the experimenter has to trust thesubjective report of the patient (as withECS mapping). What can be testedobjectively is whether the patient actuallyfound a correlation between the objectiveand a subjective marker, because s/he mustthen be able to signal the variations of theobjective marker based only on her/his ownexperience even if he has no access to thefeedback. Also, s/he must be able toidentify catch trials in which the activitythat is fed back is not real-time, butrecorded say a minute earlier. Furthermore,once a subjective event is clearlyrecognised by the subject, s/he might stillnot be able adequately to verbalise it, ormight yet succeed in fooling theexperimenter. These considerations furthermilitate for the careful design of protocolsto manipulate specifically the dimensionsreported by the patient to validate her/hisdescription.

The second limitation of theneurofeedback approach comes from thevariability of neural signals. Even ifprecautions are taken to avoid variations inthe subject’s environment, there is a high

degree of variability in neural responseseven to identical external stimulations(Arieli et al., 1996). This variability is morepronounced in recordings with a low spatialresolution, such as scalp EEG, than withmore precise recordings such as intracranialEEG, which suggests that it is partly due tothe contamination of the signal by neuralactivities not related to the repeatedexperimental process. For this reason, allthe studies reporting task-related gammaband modulations have had to rely onaveraging procedures across multiplerepetitions of the same situation. Thisobserved variability is lower in earlysensory areas or in the motor cortex. Forinstance, the activity that can be recorded inthe primary auditory cortex in human withintracranial EEG electrodes faithfullyreflects the envelope of the auditorystimulus, with li t t le noise (Liegeois-Chauvel et al., 2004). However, variabilityincreases in more associative brain regions,which may limit the interpretability of thesignal by the patient in those areas. Thecritical point is whether or not thisvariability leaves a trace in the patient’sconscious experience. It has been shownthat some of the spontaneous fluctuationsare due to the modulation of actualcognitive processes that are usually leftuncontrolled, such as attentional andemotional fluctuations or random thoughts,of which the patient can become aware of(Lutz et al., 2002). For such sources ofvariability, it is likely that the patientscould quickly learn to control and stabilizethe variability in neural activity. This wouldpredict that the variability of the markershould decrease with practice.

In conclusion, this study demonstrates thatneurofeedback sessions based on onlineDynamic Spectral Imaging can improve theidentification of eloquent brain areas inepileptic patients candidate for surgery.Further, these sessions provide a powerfulway of understanding the anatomical basis ofcognition when used in conjunction withwell-calibrated cognitive paradigms. Therecent development of new sourcereconstruction algorithms should enable on-line Dynamical Spectral Imaging with highspatial resolution also in normal subjects


recorded non-invasively with MEG(Hoogenboom et al., 2006). This paves theway for rehabilitation therapy techniques thatwould provide feedback to the subjects’neural markers of specific cognitiveprocesses, such as attention or memory, to betrained or retrained (De Charms et al., 2005).

ACKNOWLEDGEMENTS

We thank Valérie Balle, Patricia Boschetti,Carole Chatelard, Véronique Dorlin, ElianeGamblin and Martine Juillard for theirinvaluable help. We specially thank DanielleDavid for help with the experimentalprocedure and Rainer Goebel for helpfulcomments on an earlier version of themanuscript. This work is dedicated to thelate Francisco Varela (1945-2001).

REFERENCES

ARIELI A, STERKIN A, GRINVALD A, AERTSEN A(1996) Dynamics of ongoing activity: explanation ofthe large variability in evoked cortical responses.Science 273: 1868-1871

BELIN P, ZATORRE RJ, AHAD P (2002) Humantemporal-lobe response to vocal sounds. Brain ResCogn Brain Res 13: 17-26

BELIN P, ZATORRE RJ, LAFAILLE P, AHAD P, PIKE B(2000) Voice-selective areas in human auditory cortex.Nature 403: 309-312

BIDET-CAULET A, FISCHER C, BAUCHET F,BERTRAND O (2003) Multiple sources of inducedgamma oscillations in the auditory cortex observedfrom human intracranial EEG. Neuroimage 19: 1554

BODDAERT N, CHABANE N, GERVAIS H, GOOD CD,BOURGEOIS M, PLUMET MH, BARTHELEMY C,MOUREN MC, ARTIGES E, SAMSON Y,BRUNELLE F, FRACKOWIAK RS, ZILBOVICIUS M(2004) Superior temporal sulcus anatomicalabnormalities in childhood autism: a voxel-basedmorphometry MRI study. Neuroimage 23: 364-369

BROVELLI A, LACHAUX JP, KAHANE P,BOUSSAOUD D (2005) High gamma frequencyoscillatory activity dissociates attention from intentionin the human premotor cortex. Neuroimage 28: 154-164

CHATRIAN GE, BICKFORD RG, UIHLEIN A (1960)Depth electrographic study of a fast rhythm evokedfrom the human calcarin region by steady illumination.Elec Clin Neurophysiol 12: 167-176

Cohen L, Jobert A, Le Bihan D, Dehaene S (2004)DISTINCT unimodal and multimodal regions for wordprocessing in the left temporal cortex. Neuroimage 23:1256-1270

CRONE NE, BOATMAN D, GORDON B, HAO L (2001)Induced electrocorticographic gamma activity duringauditory perception. Brazier Award-winning article,2001. Clin Neurophysiol 112: 565-582

DE CHARMS RC, MAEDA F, GLOVER GH, LUDLOWD, PAULY JM, SONEJI D, GABRIELI JD, MACKEYSC (2005) Control over brain activation and painlearned by using real-time functional MRI. Proc NatlAcad Sci U S A 102: 18626-18631

DI PELLEGRINO G, FADIGA L, FOGASSI L, GALLESEV, RIZZOLATTI G (1992) Understanding motorevents: a neurophysiological study. Exp Brain Res 91:176-180

EDWARDS E, SOLTANI M, DEOUELL LY, BERGERMS, KNIGHT RT (2005) High gamma activity inresponse to deviant auditory stimuli recorded directlyfrom human cortex. J Neurophysiol 94: 4269-4280

FECTEAU S, ARMONY JL, JOANETTE Y, BELIN P(2004) Is voice processing species-specific in humanauditory cortex? An fMRI study. Neuroimage 23: 840-848

FELL J, KLAVER P, LEHNERTZ K, GRUNWALD T,SCHALLER C, ELGER CE, FERNANDEZ G (2001)Human memory formation is accompanied by rhinal-hippocampal coupling and decoupling. Nat Neurosci 4:1259-1264

FRIEDERICI AD, RUSCHEMEYER SA, HAHNE A,FIEBACH CJ (2003) The role of left inferior frontaland superior temporal cortex in sentencecomprehension: localizing syntactic and semanticprocesses. Cereb Cortex 13: 170-177

GRAIMANN B, HUGGINS JE, LEVINE SP,PFURTSCHELLER G (2004) Toward a direct braininterface based on human subdural recordings andwavelet-packet analysis. IEEE Trans Biomed Eng 51:954-962

HOOGENBOOM N, SCHOFFELEN JM, OOSTENVELDR, PARKES LM, FRIES P (2006) Localizing humanvisual gamma-band activity in frequency, time andspace. Neuroimage 29: 764-773

HOWARD MW, RIZZUTO DS, CAPLAN JB, MADSENJR, LISMAN J, ASCHENBRENNER-SCHEIBE R,SCHULZE-BONHAGE A, KAHANA MJ (2003)Gamma oscillations correlate with working memoryload in humans. Cereb Cortex 13: 1369-1374

HUMPHRIES C, LOVE T, SWINNEY D, HICKOK G(2005) Response of anterior temporal cortex tosyntactic and prosodic manipulations during sentenceprocessing. Hum Brain Mapp 26: 128-138

JACK A, ROEPSTORFF A, eds. 2004. Trusting the subject? The Use of Introspective Evidence in CognitiveScience. London: Imprint Academic

KAHANE P, HOFFMANN D, MINOTTI L, BERTHOZ A(2003) Reappraisal of the human vestibular cortex bycortical electrical stimulation study. Ann Neurol 54:615-624

KAHANE P, MINOTTI L, HOFFMANN D, LACHAUX J,RYVLIN P (2004) Invasive EEG in the definition ofthe seizure onset zone: depth electrodes. In: Handbookof Clinical Neurophysiology. Pre-surgical assessmentof the epilepsies with clinical neurophysiology andfunctional neuroimaging (Rosenow, F. and Lüders,H.O., eds.): Elsevier Science

KAHANE P, TASSI L, FRANCIONE S, HOFFMANN D,LO RUSSO G, MUNARI C (1993) Manifestationsélectro-cliniques induites par la stimulation électriqueintra-cérébrale par “chocs” dans les épilepsiestemporales. Neurophysiol Clin 22: 305-326

KIM DS, RONEN I, OLMAN C, KIM SG, UGURBIL K,TOTH LJ (2004) Spatial relationship between neuronalactivity and BOLD functional MRI. Neuroimage 21:876-885

LACHAUX JP, FONLUPT P, KAHANE P, MINOTTI L,HOFFMANN D, BERTRAND O, BACIU M (in press)


On the relationship between task-related Gammaoscillations and BOLD signal: new insights fromcombined fMRI and intracranial EEG. Human BrainMapping

LACHAUX JP, HOFFMANN D, MINOTTI L, BERTHOZA, KAHANE P (2005) Intracerebral Dynamics ofSaccade Generation in the Human Frontal Eye Fieldand Supplementary Eye Field. Neuroimage in press

LACHAUX JP, RODRIGUEZ E, MARTINERIE J, ADAMC, HASBOUN D, VARELA FJ (2000) A quantitativestudy of gamma-band activity in human intracranialrecordings triggered by visual stimuli. Eur J Neurosci12: 2608-2622

LACHAUX JP, RUDRAUF D, KAHANE P (2003)Intracranial EEG and human brain mapping. J PhysiolParis 97: 613-628

LESSER RP, ARROYO S, CRONE N, GORDON B (1998)Motor and sensory mapping of the frontal and occipitallobes. Epilepsia 39 Suppl 4: S69-80

LEUTHARDT EC, SCHALK G, WOLPAW JR,OJEMANN JG, MORAN DW (2004) A brain-computerinterface using electrocorticographic signals inhumans. J Neural Eng 1: 63-71

LIEGEOIS-CHAUVEL C, LORENZI C, TREBUCHON A,REGIS J, CHAUVEL P (2004) Temporal envelopeprocessing in the human left and right auditorycortices. Cereb Cortex 14: 731-740

LOBEL E, KAHANE P, LEONARDS U, GROSBRAS M,LEHERICY S, LE BIHAN D, BERTHOZ A (2001)Localization of human frontal eye fields: anatomicaland functional f indings of functional magneticresonance imaging and intracerebral electricalstimulation. J Neurosurg 95: 804-815

LUDERS HO, DINNER DS, MORRIS HH, WYLLIE E,COMAIR YG (1995) Cortical electrical stimulation inhumans. The negative motor areas. Adv Neurol 67:115-129

LUTZ A, LACHAUX JP, MARTINERIE J, VARELA FJ(2002) Guiding the study of brain dynamics by usingfirst-person data: synchrony patterns correlate withongoing conscious states during a simple visual task.Proc Natl Acad Sci U S A 99: 1586-1591

MAINY N, KAHANE P, MINOTTI L, HOFFMANN D,BERTRAND O, LACHAUX JP (2007) Neuralcorrelates of consolidation in working memory. HumBrain Mapp 28: 183-193

MUKAMEL R, GELBARD H, ARIELI A, HASSON U,FRIED I, MALACH R (2005) Coupling betweenneuronal firing, field potentials, and FMRI in humanauditory cortex. Science 309: 951-954

NIESSING J, EBISCH B, SCHMIDT KE, NIESSING M,SINGER W, GALUSKE RA (2005) Hemodynamicsignals correlate tightly with synchronized gammaoscillations. Science 309: 948-951

OJEMANN G, OJEMANN J, LETTICH E, BERGER M(1989) Cort ical language local izat ion in lef t ,dominant hemisphere. An electrical stimulationmapping investigation in 117 patients. J Neurosurg71: 316-326

PENFIELD W, JASPER H (1954) Epilepsy and thefunctional anatomy of the human brain. Boston: LittleBrown

PETITMENGIN C (2006) Describing one’s subjectiveexperience in the second-person. Phenomenology andthe Cognitive Sciences (in press)

PETITOT J, VARELA FJ, PACHOUD B, ROY JM, eds.1999. Naturalizing Phenomenology: ContemporaryIssues in Phenomenology and Cognitive Science.Stanford: Stanford University Press

PFURTSCHELLER G, BRUNNER C, SCHLOGL A,LOPES DA SILVA FH (2006) Mu rhythm(de)synchronization and EEG single-trial classificationof different motor imagery tasks. Neuroimage

PFURTSCHELLER G, GRAIMANN B, HUGGINS JE,LEVINE SP, SCHUH LA (2003) Spatiotemporalpatterns of beta desynchronization and gammasynchronization in corticographic data during self-paced movement. Clin Neurophysiol 114: 1226-1236

PORTNEY LG, WATKINS MP (1993) In: Foundations ofClinical Research Applications and practice, pp 430-432. London: Prentice Hall International

SCOTT SK, JOHNSRUDE IS (2003) The neuroanatomicaland functional organization of speech perception.Trends Neurosci 26: 100-107

SINAI A, BOWERS CW, CRAINICEANU CM,BOATMAN D, GORDON B, LESSER RP, LENZ FA,CRONE NE (2005) Electrocorticographic high gammaactivity versus electrical cortical stimulation mappingof naming. Brain 128: 1556-1570

SINGER W (1999) Neuronal synchrony: a versatile codefor the definition of relations? Neuron 24: 49-65, 111-125

SZURHAJ W, BOURRIEZ JL, KAHANE P, CHAUVEL P,MAUGUIERE F, DERAMBURE P (2005)Intracerebral study of gamma rhythm reactivity in thesensorimotor cortex. Eur J Neurosci 21: 1223-1235

TALAIRACH J, TOURNOUX P (1988) Co-planarstereotaxic atlas of the human brain. 3-dimensionalproportional system: an approach to cerebral imaging.New York: Thieme

TALLON-BAUDRY C, BERTRAND O (1999) Oscillatorygamma activity in humans and its role in objectrepresentation. Trends Cogn Sci 3: 151-162

TANJI K, SUZUKI K, DELORME A, SHAMOTO H,NAKASATO N (2005) High-frequency gamma-bandactivity in the basal temporal cortex during picture-naming and lexical-decision tasks. J Neurosci 25:3287-3293

THOMPSON E, VARELA FJ (2001) Radical embodiment:neural dynamics and consciousness. Trends Cogn Sci5: 418-425

VARELA F, SHEAR J, eds. 1999. The View from Within:First-Person Methodologies in the Study ofConsciousness. London: Imprint Academic

VARELA FJ (1996) Neurophenomenology: amethodological remedy for the hard problem. Journalof Consciousness Studies, 3: 330-350

VAUGHAN TM, HEETDERKS WJ, TREJO LJ, RYMERWZ, WEINRICH M, MOORE MM, KUBLER A,DOBKIN BH, BIRBAUMER N, DONCHIN E,WOLPAW EW, WOLPAW JR (2003) Brain-computerinterface technology: a review of the SecondInternational Meeting. IEEE Trans Neural Syst RehabilEng 11: 94-109

braintv: a novel approach for online mapping of human brain functions · 2009-08-03 · frequency...

Documents