introductiontocassandra(ebenhewitt,XXXX09)

introductiontocassandraebenhewittseptember29.2010web2.0exponewyorkcity•director,applicationarchitectureataglobalcorp•focusonSOA,SaaS,Events•iwrotethis@ebenhewittagenda•context•features•datamodel•api“nosql”“bigdata”•mongodb•couchdb•tokyocabinet•redis•riak•whatabout?–Poet,Lotus,Xindice–they’vebeenaroundforever…–rdbmswasoncethenewkid…innovationatscale•googlebigtable(2006)–consistencymodel:strong–datamodel:sparsemap–clones:hbase,hypertable•amazondynamo(2007)–O(1)dht–consistencymodel:clienttune-able–clones:riak,voldemortcassandra~=bigtable+dynamoproven•TheFacebookstores150TBofdataon150nodesweb2.0•usedatTwitter,Rackspace,Mahalo,Reddit,Cloudkick,Cisco,Digg,SimpleGeo,Ooyala,OpenX,otherscaptheorem•consistency–allclientshavesameviewofdata•availability–writeableinthefaceofnodefailure•partitiontolerance–processingcancontinueinthefaceofnetworkfailure(crashedrouter,brokennetwork)danielabadi:pacelcwriteconsistencyLevelDescriptionZEROGoodluckwiththatANY1replica(hintscount)ONE1replica.readrepairinbkgndQUORUM(DCQforRackAware)(N/2)+1ALLN=replicationfactorLevelDescriptionZEROUmmm…ANYTryONEinsteadONE1replicaQUORUM(DCQforRackAware)ReturnmostrecentTSafter(N/2)+1reportALLN=replicationfactorreadconsistencyagenda•context•features•datamodel•apicassandraproperties•tuneablyconsistent•veryfastwrites•highlyavailable•faulttolerant•linear,elasticscalability•decentralized/symmetric•~12clientlanguages–ThriftRPCAPI•~automaticprovisioningofnewnodes•0(1)dht•bigdatawriteopStagedEvent-DrivenArchitecture•Ageneral-purposeframeworkforhighconcurrency&loadconditioning•Decomposesapplicationsintostagesseparatedbyqueues•Adoptastructuredapproachtoevent-drivenconcurrencyinstrumentationdatareplication•configurablereplicationfactor•replicaplacementstrategyrackunawareSimpleStrategyrackawareOldNetworkTopologyStrategydatacentershardNetworkTopologyStrategypartitionersmack-downRandomPreserving•systemwilluseMD5(key)todistributedataacrossnodes•evendistributionofkeysfromoneCFacrossranges/nodesOrderPreserving•keydistributiondeterminedbytoken•lexicographicalordering•requiredforrangequeries–scanoverrowslikecursorinindex•canspecifythetokenforthisnodetouse•‘scrabble’distributionagenda•context•features•datamodel•apistructurekeyspacesettings(eg,partitioner)columnfamilysettings(eg,comparator,type[Std])columnnamevalueclockkeyspace•~=database•typicallyoneperapplication•somesettingsareconfigurableonlyperkeyspacecolumnfamily•grouprecordsofsimilarkind•notsamekind,becauseCFsaresparsetables•ex:–User–Address–Tweet–PointOfInterest–HotelRoomthinkofcassandraasrow-oriented•eachrowisuniquelyidentifiablebykey•rowsgroupcolumnsandsupercolumnscolumnfamilyn=42user=ebenkey123key456user=alisonicon=nickname=TheSituationjson-likenotationUser{123:{email:[e-mail],icon:},456:{email:[e-mail],location:TheDangerZone}}0.6example$cassandra–f$bin/cassandra-clicassandraconnectlocalhost/9160cassandrasetKeyspace1.Standard1[‘eben’][‘age’]=‘29’cassandrasetKeyspace1.Standard1[‘eben’][‘email’]=‘[e-mail]’cassandragetKeyspace1.Standard1[‘eben'][‘age']=(column=6e616d65,value=39,timestamp=1282170655390000)acolumnhas3parts1.name–byte[]–determinessortorder–usedinqueries–indexed2.value–byte[]–youdon’tqueryoncolumnvalues3.timestamp–long(clock)–lastwritewinsconflictresolutioncolumncomparators•byte•utf8•long•timeuuid•lexicaluuid•pluggable–ex:lat/longsupercolumnsupercolumnsgroupcolumnsunderacommonnameSCFPointOfInterestsupercolumnfamilySCCentralPark10017SCEmpireStateBldgSCPhoenixZoo85255desc=Funtowalkin.phone=212.555.11212desc=Greatviewfrom102ndfloor!PointOfInterest{key:85255{PhoenixZoo{phone:[phone],desc:Theyhaveanimalshere.},SpringTraining{phone:[phone],desc:Funforbaseballfans.},},//endphxkey:10019{CentralPark{desc:Walkaround.It'spretty.},EmpireStateBuilding{phone:[phone],desc:Greatviewfrom102ndfloor.}}//endnyc}ssupercolumnsupercolumnfamilyflexibleschemakeycolumnsupercolumnfamilyaboutsupercolumnfamilies•sub-columnnamesinaSCFarenotindexed–toplevelcolumns(SCFName)arealwaysindexed•oftenusedfordenormalizingdatafromstandardCFsagenda•context•features•datamodel•apislicepredicate•datastructuredescribingcolumnstoreturn–SliceRange•startcolumnname•finishcolumnname(canbeemptytostoponcount)•reverse•count(likeLIMIT)readapi•get():Column–gettheColorSCatgivenColPathCOSCcosc=client.get(key,path,CL);•get_slice():ListColumnOrSuperColumn–getColsinonerow,specifiedbySlicePredicate:ListColumnOrSuperColumnresults=client.get_slice(key,parent,predicate,CL);•multiget_slice():Mapkey,ListCoSC–getslicesforlistofkeys,basedonSlicePredicateMapbyte[],ListColumnOrSuperColumnresults=client.multiget_slice(rowKeys,parent,predicate,CL);•get_range_slices():ListKeySlice–returnsmultipleColsaccordingtoarange–rangeisstartkey,endkey,starttoken,endtoken:ListKeySliceslices=client.get_range_slices(parent,predicate,keyRange,CL);writeapiclient.insert(userKeyBytes,parent,newColumn(“band.getBytes(UTF8),“Funkadelic.getBytes(),clock),CL);batch_mutate–voidbatch_mutate(mapbyte[],mapString,ListMutation,CL)