[ORANGE] Preprocessing Menggunakan Teknik Median

PREPROCESSING MENGGUNAKAN TEKNIK MEDIAN

1.      Alasan memilih teknik median karena menurut saya, jika data kosong diisi dengan median atau nilai tengah, maka kurang lebih output yang dihasilkan menjadi setara.

2.       Langkah-langkah:

a.       Input file product.csv

b.       Membuat alur teknik processing

c.       Cek isi table sebelum dilakukan processing

d.       Klik Preprocess, kemudian atur agar data kosong terisi dengan median

e.       Cek data yang telah dilakukan processing

Read More

Identifikasi Outliers Menggunakan Jupyter

TUGAS IDENTIFIKASI OUTLIERS

      1.      Install PYOD

      2.      Import matplotlib

      3.      Membaca data

      4.      Mengecek plot kolom Hours Per Day

      5.      Mengecek batasan

      6.      Untuk mendeteksi nilai yang merupakan outliers

Read More

Data Mining NoteBook

     1.      Load Data

     import pandas as pd

     dataset=pd.read_csv("E:\KULIAH\SEMESTER 6\DATMIN\PreprocessDataSet.csv")

     2.      Menampilkan Data

a.      dimensi data

print(dataset.shape)

b.      5 data pertama

print(dataset.head(5))

c.       5 data terakhir

print(dataset.tail(5))

d.      semua data

print(dataset)

e.       ringkasan data yang (integer)

print(dataset.describe())

f.        ringkasan data (tipe kolom mix)

print(dataset.describe(include = 'all'))

g.      jumlah missing value

dataset.isnull().sum()

h.      menampilkan, pada baris keberapa, dataset pada suatu kolom, mengandung missing value

dataset[dataset.Marital_Status.isnull()]

     3.      count   : menghitung jumlah baris yang gak missing

     unique : isinya apa aja (misal gender, ada male dan female)

     top/modus: data yang paling sering muncul

     freq      : berapa byk / jumlah yg sering muncul (missal ‘yahoo’ frekuensinya 6)

     mean    : nilai rata-rata

     std       : (standart defiasi) 

     min      : nilai minimum

     max     : nilai terbesar

     4.      Periksa dimensi sebelum remove

a.      jika semua baris yang mengandung missing value di kolom manapun, didrop

dataset.dropna(how='any').shape

b.      jika semua baris yang mengandung missing value di semua kolom didrop

dataset.dropna(how='all').shape

c.       jika semua baris yang mengandung missing value di kolom tertentu didrop (any [salah 1] /all [harus semua yg salah])

·         dataset.dropna(subset=['Marital_Status', 'Online_Gaming'], how='any').shape

·         dataset.dropna(subset=['Marital_Status', 'Online_Gaming'], how='all').shape

     5.      Mengganti nilai dengan random

a.      memeriksa  frekuensi sebaran nilai pada kolom tertentu

dataset['Marital_Status'].value_counts()   à munculnya S (4), D (3), M (3)

b.      mereplace missing value dengan nilai tertentu

import numpy as np

dataset['Marital_Status'].replace(np.nan, 'S', inplace=True)   à memilih mengganti dengan S krn modus

     6.     Cek lagi datanya

      print(dataset)

Read More

Data Mining (Klasifikasi)

   1.      Import library yang diperlukan

   %matplotlib inline

   import numpy as np

   import scipy as sp

   import matplotlib as mpl

   import matplotlib.cm as cm

   import matplotlib.pyplot as plt

   import pandas as pd

   from pandas.tools.plotting import scatter_matrix

   pd.set_option('display.width',500)

   pd.set_option('display.max_columns',100)

   pd.set_option('display.notebook_repr_html',True)

   import seaborn as sns

   sns.set(style='whitegrid')

   from sklearn.tree import DecisionTreeClassifier

   from sklearn import metrics

   from sklearn.model_selection import train_test_split

   2.      Load Data Set

   data= pd.read_csv('E:/KULIAH/SEMESTER 6/DATMIN/Materi/train.csv')

   Cek Data Set

   data.head()

   3.      Pemahaman data (EDA) dan preprocess data

   ·         Statistik

   data.describe()

   ·         Visual

   data["Survived"].value_counts().plot(kind="bar")

   data["Survived"].value_counts()

   4.      Perbandingan survival rated engan variable lainnya

         a.      Buat fungsi untuk menghitung dan menampilkan grafiknya

         def survival_stacked_bar(variable):

         Died=data[data["Survived"]==0][variable].value_counts()/len(data["Survived"]==0)

         Survived=data[data["Survived"]==1][variable].value_counts()/len(data["Survived"]==1)

         data=pd.DataFrame([Died,Survived])

         data.index=["Did not survived", "Survived"]

         data.plot(kind="bar",stacked=True,title="Percentage")

         return data.head()

        b.      Panggil fungsi tersebut

        survival_stacked_bar("Sex")

   5.      Preprocess Data

         a.       Missing Value? Cek apakah ada missing value

          data.isnull().sum()

         b.      Membuat stacked bar

          def titanic_corr(data):

          correlation = data.corr()

          sns.heatmap(correlation, annot=True, cbar=True, cmap="RdYlGn")

         c.       Melihat stacked bar

          titanic_corr(data)

         d.      Menambah kolom ‘Title’ untuk memberi tingkatan pada gelar penumpang

          data.insert(value=data.Name.map(lambda name: name.split(",")[1].split(".")[0].strip()),loc=12,column="Title")

          e.       Isi kolom ‘Title’

          title_map={'Capt':1,

                  'Col': 1,

                  'Mayor': 1,

                  'Jonkheer': 2,

                  'Don': 2,

                  'Sir': 2,

                  'Dr': 3,

                  'Rev': 3,

                  'the Countess': 2,

                  'Dona': 2,

                  'Mme': 4,

                  'Mlle': 5,

                  'Ms': 4,

                  'Mr': 6,

                  'Mrs': 7,

                 ' Miss': 8,

                 'Master': 9,

                 'Lady': 2,}

         data["Title"]=data.Title.map(title_map)

        f.        Cek data setelah ditambahkan kolom ‘Title’

         data

         g.      Mengubah jenis kelamin menjadi angka 0 dan 1

         sex_map={"male":1,"female":0}

         data["Sex"]=data["Sex"].map(sex_map)

         h.      Cek data setelah melakukan perubahan

         data

Read More

KONEK DATABASE DAN LANGKAH-LANGKAH VB

KONEK DATABASE

1. sqlpuls/nolog
2. conn sys/oracle@orcl as sysdba;
3. create user audy identified by audy;
4. grant connect to audy;
5. conn audy/audy@orcl
6. conn sys/oracle@orcl as sysdba;
7. grant all privileges to audy identified by audy;
8. conn audy/audy@orcl
9. create table mhw (npm char(10), nama varchar2(50), alamat varchar2(50));
10. insert into mhw (npm, nama, alamat) values (1635010004, 'erica', 'lmg');
11. select * from mhw;
12. commit;

VB CONNECTED

1. new project
2. add reference --> sys data oracle client
3. tools --> connect to db
4. toolbox --> listview --> properties --> coloumn (collection) yg ... klik
   add --> text diganti nama kolom npm
   add --> text diganti nama kolom nama
   add --> text diganti nama kolom alamat
   kalau tidak muncul, properties --> view diganti details
5. import sys
6. inisialisasi Dim
   Dim xconn
   Dim xcomm
   Dim dr
   Dim npm, nama, alamat As String
7. buat sub konek --> pakai:
   conn close()
   conn open()
   Ditaruh diatas form load
8. manggil konek

VB DISCONNECTED

1. new project
2. add reference --> sys data oracle client
3. tools --> connect to db
4. toolbox --> DataGridView
5. klik2x utk koding
6. import sys
7. Dim, dkk
8. sub koneksi --> end sub
9. server explore --> properties
10. private sub
11. ganti 32 ke 64 + password
12. klik toolbox buat textbox + button
13. buat label, dll
14. insert into '" & .......... dll

Read More

CONNECTED

PUNYA JINJIN

Imports System.Data.OracleClient

Public Class Form1

    Dim xconn As New OracleConnection

    Dim xcomm As New OracleCommand

    Dim dr As OracleDataReader

    Dim npm, nama, alamat As String

    'KONEKSI

    Sub konek()

        xconn.Close()

        xconn = New OracleConnection("data source=ORCL; user id=jinjin; password=jinjin;")

        xconn.Open()

    End Sub

    'TAMPILAN LISTVIEW

    Private Sub Form1_Load(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles MyBase.Load

        konek()

        ListView1.Items.Clear()

        xcomm = New OracleCommand("SELECT npm, nama, alamat from mhs ", xconn)

        dr = xcomm.ExecuteReader()

        If dr.HasRows Then

            While dr.Read()

                ListView1.Items.Add(dr.Item(0))

                ListView1.Items(ListView1.Items.Count - 1).SubItems.Add(dr.Item(1))

                ListView1.Items(ListView1.Items.Count - 1).SubItems.Add(dr.Item(2))

            End While

            dr.Close()

        End If

        dr.Close()

        xcomm.Dispose()

    End Sub

    'SIMPAN

    Private Sub Button1_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Button1.Click

        xcomm = New OracleCommand("insert into mhs values('" & TextBox1.Text & "','" & TextBox2.Text & "','" & RichTextBox1.Text & "')", xconn)

        xcomm.ExecuteNonQuery()

        ListView1.Items.Clear()

        xcomm = New OracleCommand("select * from mhs", xconn)

        dr = xcomm.ExecuteReader()

        TextBox1.Text = ""

        TextBox2.Text = ""

        RichTextBox1.Text = ""

        If dr.HasRows Then

            While dr.Read()

                ListView1.Items.Add(dr.Item(0))

                ListView1.Items(ListView1.Items.Count - 1).SubItems.Add(dr.Item(1))

                ListView1.Items(ListView1.Items.Count - 1).SubItems.Add(dr.Item(2))

            End While

            dr.Close()

        End If

        dr.Close()

        xcomm.Dispose()

    End Sub

    'CARI

    Private Sub Button2_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Button2.Click

        xcomm = New OracleCommand("select * from mhs where npm = '" & TextBox1.Text & "'", xconn)

        dr = xcomm.ExecuteReader

        If dr.HasRows Then

            While dr.Read

                TextBox1.Text = dr.Item(0)

                TextBox2.Text = dr.Item(1)

                RichTextBox1.Text = dr.Item(2)

            End While

        End If

    End Sub

    'UPDATE

    Private Sub Button3_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Button3.Click

        xcomm = New OracleCommand("update mhs set nama='" & TextBox2.Text & "',alamat='" & RichTextBox1.Text & "' where npm = '" & TextBox1.Text & "'", xconn)

        xcomm.ExecuteNonQuery()

        ListView1.Items.Clear()

        xcomm = New OracleCommand("select * from mhs", xconn)

        dr = xcomm.ExecuteReader()

        If dr.HasRows Then

            While dr.Read()

                ListView1.Items.Add(dr.Item(0))

                ListView1.Items(ListView1.Items.Count - 1).SubItems.Add(dr.Item(1))

                ListView1.Items(ListView1.Items.Count - 1).SubItems.Add(dr.Item(2))

            End While

            dr.Close()

        End If

        dr.Close()

        xcomm.Dispose()

    End Sub

    'DELETE

    Private Sub Button4_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Button4.Click

        xcomm = New OracleCommand("delete mhs where npm='" & TextBox1.Text & "'", xconn)

        xcomm.ExecuteNonQuery()

        ListView1.Items.Clear()

        xcomm = New OracleCommand("select * from mhs", xconn)

        dr = xcomm.ExecuteReader()

        If dr.HasRows Then

            While dr.Read()

                ListView1.Items.Add(dr.Item(0))

                ListView1.Items(ListView1.Items.Count - 1).SubItems.Add(dr.Item(1))

                ListView1.Items(ListView1.Items.Count - 1).SubItems.Add(dr.Item(2))

            End While

            dr.Close()

        End If

        dr.Close()

        xcomm.Dispose()

    End Sub

End Class

PUNYA GEJROT

Imports System.Data.OracleClient

Public Class Form1

    Dim xconn As New OracleConnection

    Dim xcomm As New OracleCommand

    Dim dr As OracleDataReader

    Dim da As New OracleDataAdapter

    Dim ds As New DataSet

    Dim ctr As Control

    Sub koneksi()

        xconn.Close()

        xconn.ConnectionString = "Data Source=ORCL;User ID=tantra;password=tantra;Unicode=True"

        xconn.Open()

    End Sub

    Private Sub Form1_Load(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles MyBase.Load

        koneksi()

        ListView1.Items.Clear()

        xcomm = New OracleCommand("SELECT kode_barang, nama_barang, jenis, satuan, harga_beli, harga_jual, stock from tblbarang ", xconn)

        dr = xcomm.ExecuteReader()

        If dr.HasRows Then

            While dr.Read()

                ListView1.Items.Add(dr.Item(0))

                ListView1.Items(ListView1.Items.Count - 1).SubItems.Add(dr.Item(1))

                ListView1.Items(ListView1.Items.Count - 1).SubItems.Add(dr.Item(2))

                ListView1.Items(ListView1.Items.Count - 1).SubItems.Add(dr.Item(3))

                ListView1.Items(ListView1.Items.Count - 1).SubItems.Add(dr.Item(4))

                ListView1.Items(ListView1.Items.Count - 1).SubItems.Add(dr.Item(5))

                ListView1.Items(ListView1.Items.Count - 1).SubItems.Add(dr.Item(6))

            End While

            dr.Close()

        End If

        dr.Close()

        xcomm.Dispose()

    End Sub

    Private Sub Label1_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Label1.Click

    End Sub

    Private Sub TextBox1_TextChanged(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles kdbrg.TextChanged

    End Sub

    Private Sub ListView1_SelectedIndexChanged(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles ListView1.SelectedIndexChanged

    End Sub

    Private Sub Button5_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles btnhapus.Click

    End Sub

    Private Sub Label7_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Label7.Click

    End Sub

    Private Sub btnbr_Click(ByVal sender As System.Object, ByVal e As System.EventArgs)Handles btnbr.Click

        '''''sintak untuk membersihkan textbox dan NumericUpDown

        For Each ctr In Me.Controls

            If TypeOf ctr Is TextBox Then

                ctr.Text = ""

            End If

            If TypeOf ctr Is NumericUpDown Then

                ctr.Text = "0"

            End If

        Next

    End Sub

    Private Sub Btnsimpan_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles btnsimpan.Click

        '''''digunakan untuk sintak DML, untuk button ini digunakan untuk insert data

        xcomm = New OracleCommand("insert into tblbarang values ('" & kdbrg.Text & "','" & nmbrg.Text & "'," & "'" & jns.Text & "','" & satuan.Text & "'," & HB.Value & "," & HJ.Value & "," & STK.Value & ")", xconn)

        xcomm.ExecuteNonQuery()

        '''' digunakan untuk menampilkan data ke listview setelah memasukkan data

        ListView1.Items.Clear()

        xcomm = New OracleCommand("SELECT kode_barang, nama_barang, jenis, satuan, harga_beli, harga_jual, stock from tblbarang", xconn)

        dr = xcomm.ExecuteReader()

        If dr.HasRows Then

            While dr.Read()

                ListView1.Items.Add(dr.Item(0))

                ListView1.Items(ListView1.Items.Count - 1).SubItems.Add(dr.Item(1))

                ListView1.Items(ListView1.Items.Count - 1).SubItems.Add(dr.Item(2))

                ListView1.Items(ListView1.Items.Count - 1).SubItems.Add(dr.Item(3))

                ListView1.Items(ListView1.Items.Count - 1).SubItems.Add(dr.Item(4))

                ListView1.Items(ListView1.Items.Count - 1).SubItems.Add(dr.Item(5))

                ListView1.Items(ListView1.Items.Count - 1).SubItems.Add(dr.Item(6))

            End While

            dr.Close()

        End If

        dr.Close()

        xcomm.Dispose()

    End Sub

 

End Class

Imports System.Data.OracleClient

Public Class Form1

    Dim xconn As New OracleConnection

    Dim xcomm As New OracleCommand

    Dim dr As OracleDataReader

    Dim da As New OracleDataAdapter

    Dim ds As New DataSet

    Dim ctr As Control

    Sub koneksi()

        xconn.Close()

        xconn.ConnectionString = "Data Source=ORCL;User ID=tantra;password=tantra;Unicode=True"

        xconn.Open()

    End Sub

    Private Sub Form1_Load(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles MyBase.Load

        koneksi()

        ListView1.Items.Clear()

        xcomm = New OracleCommand("SELECT kode_barang, nama_barang, jenis, satuan, harga_beli, harga_jual, stock from tblbarang ", xconn)

        dr = xcomm.ExecuteReader()

        If dr.HasRows Then

            While dr.Read()

                ListView1.Items.Add(dr.Item(0))

                ListView1.Items(ListView1.Items.Count - 1).SubItems.Add(dr.Item(1))

                ListView1.Items(ListView1.Items.Count - 1).SubItems.Add(dr.Item(2))

                ListView1.Items(ListView1.Items.Count - 1).SubItems.Add(dr.Item(3))

                ListView1.Items(ListView1.Items.Count - 1).SubItems.Add(dr.Item(4))

                ListView1.Items(ListView1.Items.Count - 1).SubItems.Add(dr.Item(5))

                ListView1.Items(ListView1.Items.Count - 1).SubItems.Add(dr.Item(6))

            End While

            dr.Close()

        End If

        dr.Close()

        xcomm.Dispose()

    End Sub

    Private Sub Label1_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Label1.Click

    End Sub

    Private Sub TextBox1_TextChanged(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles kdbrg.TextChanged

    End Sub

    Private Sub ListView1_SelectedIndexChanged(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles ListView1.SelectedIndexChanged

    End Sub

    Private Sub Button5_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles btnhapus.Click

    End Sub

    Private Sub Label7_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Label7.Click

    End Sub

    Private Sub btnbr_Click(ByVal sender As System.Object, ByVal e As System.EventArgs)Handles btnbr.Click

        '''''sintak untuk membersihkan textbox dan NumericUpDown

        For Each ctr In Me.Controls

            If TypeOf ctr Is TextBox Then

                ctr.Text = ""

            End If

            If TypeOf ctr Is NumericUpDown Then

                ctr.Text = "0"

            End If

        Next

    End Sub

    Private Sub Btnsimpan_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles btnsimpan.Click

        '''''digunakan untuk sintak DML, untuk button ini digunakan untuk insert data

        xcomm = New OracleCommand("insert into tblbarang values ('" & kdbrg.Text & "','" & nmbrg.Text & "'," & "'" & jns.Text & "','" & satuan.Text & "'," & HB.Value & "," & HJ.Value & "," & STK.Value & ")", xconn)

        xcomm.ExecuteNonQuery()

        '''' digunakan untuk menampilkan data ke listview setelah memasukkan data

        ListView1.Items.Clear()

        xcomm = New OracleCommand("SELECT kode_barang, nama_barang, jenis, satuan, harga_beli, harga_jual, stock from tblbarang", xconn)

        dr = xcomm.ExecuteReader()

        If dr.HasRows Then

            While dr.Read()

                ListView1.Items.Add(dr.Item(0))

                ListView1.Items(ListView1.Items.Count - 1).SubItems.Add(dr.Item(1))

                ListView1.Items(ListView1.Items.Count - 1).SubItems.Add(dr.Item(2))

                ListView1.Items(ListView1.Items.Count - 1).SubItems.Add(dr.Item(3))

                ListView1.Items(ListView1.Items.Count - 1).SubItems.Add(dr.Item(4))

                ListView1.Items(ListView1.Items.Count - 1).SubItems.Add(dr.Item(5))

                ListView1.Items(ListView1.Items.Count - 1).SubItems.Add(dr.Item(6))

            End While

            dr.Close()

        End If

        dr.Close()

        xcomm.Dispose()

    End Sub

 

End Class

Read More