That's pretty straight forward. You'll need to make a reference to a CSS file within your manifest (uncomment the css resources line below):
<?xml version="1.0" encoding="utf-8" ?>
<manifest>
<control namespace="ControlsAndrewLy" constructor="TestProject1" version="0.0.1" display-name-key="TestProject1_Display_Key" description-key="TestProject1_Desc_Key" control-type="standard">
<!-- property node identifies a specific, configurable piece of data that the control expects from CDS -->
<property name="sampleProperty" display-name-key="Property_Display_Key" description-key="Property_Desc_Key" of-type="SingleLine.Text" usage="bound" required="true" />
<!--
Property node's of-type attribute can be of-type-group attribute.
Example:
<type-group name="numbers">
<type>Whole.None</type>
<type>Currency</type>
<type>FP</type>
<type>Decimal</type>
</type-group>
<property name="sampleProperty" display-name-key="Property_Display_Key" description-key="Property_Desc_Key" of-type-group="numbers" usage="bound" required="true" />
-->
<resources>
<code path="index.ts" order="1"/>
<!-- UNCOMMENT TO ADD MORE RESOURCES
<css path="css/TestProject1.css" order="1" />
<resx path="strings/TestProject1.1033.resx" version="1.0.0" />
-->
</resources>
</control>
</manifest>
Then, create the css folder with css file.
Each control you create (with project.ts file), will need a class name. E.g. Label has class name of YourLabelClass.
Check out some of my projects for examples:
https://github.com/365lyf/PCFControls
